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ABSTRACT 



In order to measure their effectiveness in promoting rapid 
re-employment and shorter spells during which claimants are entitled to 
collect unemployment insurance (UI) , the U.S. Department of Labor tested 
these three assistance strategies in the District of Columbia (8,071 
claimants selected during June 1995-June 1996) and Florida (12,042 claimants 
selected during March 1995-March 1996) : (1) Structured Job Search Assistance 

(SJSA) , which required orientation, testing, job search workshops, and a 
one-on-one assessment interview; (2) Individualized Job Search Assistance 
(IJSA) , which provided services based on needs of claimants; and (3) 
Individualized Job Search Assistance with Training ( I JSA+) , which provided 
the services of IJSA plus training provided by other agencies for interested 
claimants. Claimants were assigned at random to one of the three treatment 
groups or to a control group. Among the findings are the following: (1) 

compared to the control group, all treatments reduced UI receipts for the 
first year, with the largest impact occurring in the Structured Job Search 
Assistance group in the District of Columbia, where it was shortened by more 
than a week; (2) no treatments reduced UI receipt beyond the first year; (3) 
the researchers were not able to definitively answer which treatment was most 
effective in reducing UI spells; (4) the treatments had uneven impacts on 
employment and earnings; and (5) compared to the control group, all 
treatments led claimants to contact more employers per week in their job 
search. (CML) 
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EXECUTIVE SUMMARY 



BACKGROUND 

The Emergency Unemployment Compensation Act of 1991 authorized the U.S. Department of Labor 
(DOL) to conduct the Job Search Assistance (JSA) demonstration to test the feasibility of implementing 
job search assistance programs and measure their effectiveness in promoting rapid re-employment and 
reduced UI spells among Unemployment Insurance (UI) claimants. The demonstration was designed to 
identify UI claimants, at an early point of contact with the UI system, who were likely to face lengthy UI 
spells and to provide them with assistance in finding a new job. Three different assistance strategies, which 
are described below, were tested in the demonstration. In each case, assistance was provided in the 
demonstration by the local Job Service (JS) agencies, while the UI agencies monitored participation in the 
demonstration and sanctioned claimants who failed to comply with the demonstration’s participation 
requirements. 

The demonstration was conducted in the District of Columbia (D.C.) and Florida, which were chosen 
based on plans submitted to DOL. The D.C. demonstration operated in a single office and served a 
targeted sample of claimants from the full D.C. claimant population. Claimant selection occurred between 
June 1995 and June 1996, and a total of 8,071 claimants were assigned to the demonstration. The Florida 
demonstration operated in 10 local Jobs and Benefits offices scattered throughout the state. Each local 
office served a targeted sample of claimants from the local UI claimant population. Claimant selection 
occurred between March 1995 and March 1996, and a total of 12,042 claimants were assigned to the 
demonstration. 

DESIGN OF THE DEMONSTRATION 

The demonstration tested three service strategies for promoting rapid re-employment and reduced UI 
spells among targeted UI claimants: 

1 . Structured Job Search Assistance (SJSA). Claimants assigned to this treatment were required 
to participate in an orientation, testing, a job search workshop, and a one-on-one assessment 
interview. Claimants who failed to participate in any service, unless explicitly excused, could be 
denied benefits. After completion of the services, claimants were required to have two additional 
contacts with demonstration staff to report on their job search progress. 

2. Individualized Job Search Assistance (IJSA). This treatment assigned claimants to services 
based on their assessed needs. All claimants were required to participate in an orientation and a 
one-on-one assessment interview. During the assessment interview, the claimant and a 
demonstration staff member developed a service plan to address the claimant’s needs. If the 
service plan included demonstration-specific services, such as testing, a job search workshop, or 
additional counseling, these services would become mandatory. 
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3. Individualized Job Search Assistance With Training (IJSA+). This treatment was identical 
to the second treatment, except for the inclusion of a coordinated effort with local Economic 
Dislocation and Worker Adjustment Act (EDWAA) staff to enroll interested claimants in training. 
During the orientation, an EDWAA staff member discussed local opportunities for training. 
Training opportunities were also discussed during the assessment interview, and any claimant 
interested in training was scheduled to meet with an EDWAA staff member at the demonstration 
office. 

The demonstration services were intended for claimants with the greatest need for assistance-those 
expected to face long unemployment spells. Eligibility for the demonstration was determined through a two- 
stage process designed to identify such claimants. In the first stage, a series of characteristic screens was 
used to exclude claimants for whom JSA services were inappropriate, including claimants with an expected 
date of recall to their previous employer and those using union hiring halls. In the second stage, the 
probability of UI benefit exhaustion was estimated, based on a regression model, for each of the claimants 
passing the screens in the first stage. Among those who passed the screens, claimants with the highest 
exhaustion probabilities were targeted for the demonstration. 

Claimants targeted for participation were assigned randomly to a control group or to one of the three 
treatment groups. Random assignment ensures that the treatment and control groups exhibit similar 
characteristics and that each is representative, of the target population. In this demonstration, random 
assignment also allowed the differences in outcomes between each of the treatment groups and the control 
group to be interpreted as unbiased estimates of the net effects of the three service packages. 

IMPLEMENTATION OF THE DEMONSTRATION 

The demonstration was, for the most part, successfully implemented in both D.C. and Florida. Both 
states were successful in using the two-stage selection process to target the demonstration services to 
claimants likely to face long UI spells. Claimants selected as eligible for the demonstration but denied 
services (the control group) had longer average UI spells and were more likely to exhaust their benefits than 
claimants determined to be ineligible for the demonstration. In Florida, for example, the benefit exhaustion 
rate was about six percentage points higher for the demonstration-eligible claimants than for the ineligible 
claimants. In D.C., the exhaustion rate was about 13 percentage points higher for the eligible claimants. 
The differences in average UI spells between eligible and ineligible claimants was two weeks in Florida and 
1.5 weeks in D.C. These differences are not huge, but they are probably what would be expected from 
a statistical model of the determinants of benefit exhaustion among UI claimants. 

Both states generally offered the services as they were designed for each of the treatments. Claimants 
assigned to SJSA were offered a set of mandatory services, including an orientation, testing, job search 
workshop, and an assessment. The majority of claimants assigned to the demonstration attended at least 
the orientation, and the majority of those attending the orientation also attended testing, the workshop, and 
the assessment. Most of the claimants who failed to attend a service did so because they had become 
reemployed and/or had stopped collecting UI benefits. 
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Claimants assigned to IJSA and USA+ were also offered the full set of services, but few of these 
claimants participated in any JSA group services other than the orientation and assessment. Orientation 
and assessment were the only mandatory services for all claimants assigned to IJSA and IJSA+ who 
wanted to continue collecting benefits. These claimants were required to participate in the other JSA 
services-testing and the workshop-only if these services were part of the individual service plan created 
as part of their assessment interview. Few claimants in either state participated in testing or the workshop. 
Attendance was especially low in D.C., where less than 1 percent of claimants who attended orientation 
also attended testing or the workshop. The corresponding attendance rates in Florida were higher but still 
modest-in the 10 to 20 percent range. 

The low attendance rates for testing and the job search workshop in the IJSA and IJSA+ treatments 
suggest that demonstration claimants were reluctant to participate in services that were not universally 
mandatory. Furthermore, although IJSA and USA+ claimants were offered testing and the workshop, JSA 
staff were reluctant to make these services mandatory. Presumably, caseworkers either felt the services 
were inappropriate for most claimants or did not want to jeopardize claimants’ benefits by requiring 
participation. Given this, the services received by SJSA claimants, who were automatically required to 
participate in all the group services, clearly exceeded those received by the USA and USA+ claimants. 

To generate substantial rates of participation in group services, an ongoing program would probably 
need to make these services universally mandatory. Claimants are reluctant to volunteer for services, as 
has been shown in this and other demonstration evaluations. In addition, staff are unlikely to aggressively 
assign claimants to services. Given the reluctance ofboth claimants and staff to initiate service participation, 
the only way to ensure participation is to make the services mandatory. 

D.C. emphasized individual counseling rather than group services for USA and USA+ claimants. 
Nearly half of the claimants assigned to USA or IJSA+ were reported to have participated in counseling. 
The emphasis on individual counseling in D.C. may have arisen because of the limited space and trained 
staff available to conduct group services in the D.C. office. 

The timing of JSA services was consistent with the demonstration design, which was based on the 
objective of achieving early intervention. During the design phase of the demonstration, it was determined, 
given the time needed to identify and notify claimants, that services would ideally begin about 7 weeks after 
the initial UI claim. Our findings on timing of participation show that the demonstration generally achieved 
early intervention according to this standard-the average time from the beginning of the benefit year to 
orientation was about 7 weeks in both states, and about 80 to 85 percent of claimants participated in 
orientation within 8 weeks of the beginning of the benefit year. Most claimants also moved on quickly to 
subsequent services. Claimants assigned to SJSA typically finished all services by the end of the second 
full week after their orientation. Claimants assigned to USA or USA+ usually completed assessment within 
a week of their orientation. 

Data on EDWAA training show that few demonstration claimants, even those assigned to USA+, 
participated in EDWAA training. However, the training rate was higher among the combined treatment 
groups than in the control group. In Florida, the training rate was 3.5 percent for the combined treatment 
groups compared with 2.8 for the control group, while in D.C. the corresponding training rates were 1.3 



percent and 0.8 percent. These numbers imply that the information provided through orientation and 
assessment, the services offered in all treatments, was effective in inducing claimants to participate in 
EDWAA training. The effect, however, was fairly small and the resulting rate of training participation was 
also small. 

The training rate was, however, no higher among the USA+ groups than among the other treatment 
groups, so we conclude that the IJSA+ approach was not any more effective than SJSA or USA in 
providing greater access to training. There were at least two reasons for this outcome. First, demonstration 
claimants were not treated as being automatically eligible for EDWAA, as was expected when the 
demonstration was designed. In most sites, before claimants could enter EDWAA training, they had 
several eligibility or procedural hurdles to clear, which greatly impeded their potential entry into training. 
Second, coordination between the local demonstration sites and EDWAA often fell short of our 
expectations. EDWAA staff did not always participate in the demonstration services as they were 
designed, so in some local offices USA+ provided no greater contact with EDWAA than the other 
treatment groups. 

Based on the findings from previous JSA demonstration reports, we know that both states monitored 
and enforced the JSA participation requirements, but staff in the two states differed in their attitude and 
approach. Demonstration staff in both states told claimants that participation in the demonstration was 
mandatory and that claimants could lose their benefits if they refused to participate, but staff in Florida 
tended to downplay these aspects of the demonstration. Although staff in both states contacted claimants 
who failed to attend required services, D.C. tended to be more rigorous than the Florida sites in enforcing 
the requirements. In D.C., claimants who missed a single service were sent a noncompliance notice instead 
of their UI check, and they were required to report to the demonstration office to meet with a claims 
examiner to collect their benefit check. In contrast, most Florida offices allowed no-shows to maintain their 
benefits and reschedule missed services over the phone rather than by reporting to the demonstration office 
in person. Benefit checks in Florida were held up only if claimants missed multiple appointments. 

These findings demonstrate that states are likely to enforce similar participation requirements very 
differently. States will come to different decisions about what constitutes noncompliance and how to warn 
claimants that they are at risk of losing benefits. 

IMPACTS OF THE JSA DEMONSTRATION 

We estimated impacts of each of the demonstration treatments on various measures of UI receipt, 
benefit nonmonetary determinations and denials, employment and earnings, job characteristics, and job 
search activities. The treatments were expected to increase search effort, speed re-employment, and 
reduce UI benefits. 

Impacts on UI Receipt and Eligibility 

Each of the JSA treatments reduced UI receipt in the initial benefit year (year 1). The largest impact 
occurred in the SJSA group in D.C., where UI receipt was reduced by more than a week, as shown in 



0 

ERIC 



XX 



18 



Table 1. The other treatments in D.C. and all three treatments in Florida had more modest impacts, 
reducing UI receipt by about half a week. The treatments also reduced the percentage of 

TABLE 1 

ESTIMATED IMPACTS OF THE JSA TREATMENTS ON UI RECEIPT 





District of Columbia 




Florida 




Outcome 


SJSA 


IJSA 


IJSA+ 


SJSA 


IJSA 


IJSA+ 


Year 1 UI Outcomes 8 


- 












Weeks of UI Benefits 


-1.13*** 


-0.47** 


-0.61** 


-0.41** 


-0.59*** 


-0.52** 


Rate of UI Benefit Exhaustion 
(Percent) 


-4.8*** 


-2.4* 


-3.9*** 


-1.8* 


-2.4** . 


-2.8** 


Percent with at Least One Nonmonetary 
Benefit Determination 


36.6*** 


29.0*** 


28.7*** 


4.4*** 


2.7** 


2.8*** 


Percent with at Least One Nonmonetary 
Benefit Denial 


10.8*** 


8.1*** 


7.0*** 


2.9*** 


3.0*** 


2.0*** 


Year 2 UI Outcome 5 














Weeks of UI Benefits 


0.12 


-0.15 


-0.06 


0.11 


0.03 


0.17 



“Year 1 is the initial benefit year. 

b Year 2 includes all UI receipt resulting from an initial claim filed within one year of the end of year 1. 

* Statistically significant at the 90 percent level in a one-tailed test. 

"'"'Statistically significant at the 95 percent level in a one-tailed test. ■ - 

"""Statistically significant at the 99 percent level in a one-tailed test. ” 



claimants who exhausted their benefits, with the estimated reduction ranging from about 1.8 to 4.8 
percentage points. In studying the timing of the impacts on UI receipt, we found that the treatment-control 
differences in UI exit rates occurred early in UI spells, around the time that claimants were notified of JSA 
service requirements or when they would have been scheduled to participate in services. This finding 
implies that much of the impact on UI receipt is due to an immediate response to the participation 
requirements or the services rather than to a gradual application of the skills learned during program 
participation. 

None of the treatments had a significant impact on UI receipt beyond the initial benefit year. This 
finding is consistent with our expectations-we expected the treatments to help claimants become re- 
employed more quickly but not to have an effect on longer-term job stability. At the same time, our findings 
are inconsistent with those from the New Jersey UI Re-employment Demonstration, which showed that 
a structured JSA package generated a significant reduction in UI receipt in the second year after the initial 
claim. 



O 

ERIC 



XXI 



19 



These findings do not allow us to draw a definitive conclusion about which service strategy is most 
effective in reducing UI spells. In D.C., SJSA generated a larger reduction in UI spells than did USA and 
USA+. But in Florida, the impact of SJSA was nearly identical to the impacts of USA and IJSA+. Given 
these findings and our information about the enforcement policies in the two states, we conclude that the 
SJSA service approach is likely to generate larger UI reductions in settings where the additional 
participation requirements associated with SJSA are strictly enforced. Another factor that may have 
contributed to the laige impact of SJSA in D.C. is that the D.C. control group had long UI spells, so there 
was the potential for substantial reduction in UI spells in response to the demonstration. The SJSA 
approach implemented in D.C. may be an effective strategy for realizing this potential. 

All of the JSA treatments increased nonmonetary benefit eligibility determinations and denials in year 
1. Table 1 shows that the treatments in D.C. increased the rate of determination by 29 to 37 percentage 
points and the rate of denials by 7 to 1 1 percentage points, depending on the treatment. The impacts on 
determinations and denials in Florida tended to be smaller but still substantial. Most of the increases in 
determinations and denials were related to regular UI benefit eligibility issues, not to JSA participation 
directly. It appears that local staff used the information gathered through the demonstration to more strictly 
enforce traditional UI eligibility requirements for claimants assigned to the treatments. The increase in 
benefit denials was responsible for part but not all of the impact of the treatments on UI receipt, especially 
in D.C., where the increase in benefit denials was largest. 

Impacts on Employment and Earnings 

The JSA treatments had somewhat uneven impacts on employment and earnings following the initial 
UI claim. On the one hand, die SJSA group in D.C. generally had higher earnings than the control group, 
and the differences tend to be statistically significant. Moreover, the impacts of SJSA on quarterly earnings 
in D.C. were fairly large, about $200 per quarter, and persistent over the. 1 0- quarter follow-up period (see 
Table 2). On the other hand, the estimated impacts of SJSA on quarterly earnings in Florida and the 
estimated impacts of USA and IJSA+ on quarterly earnings in both states tended to be smaller (often even 
negative in Florida) and not statistically significant in most cases. However, both USA and USA+ 
significantly increased quarterly earnings in D.C. during or shortly after the initial benefit year. The impacts 
on employment rates (not shown in Table 2) were similar to the impacts on earnings. 

We found no evidence that the treatments pushed claimants into lower-quality jobs in order to hasten 
their re-employment. On the contrary, the treatments appear to have potentially improved the quality of 
the jobs accepted by participants. The treatments also did not affect the likelihood that claimants would 
switch occupations when they accepted a new job. 




xxii 



20 



TABLE 2 



ESTIMATED IMPACTS OF THE JSA TREATMENTS ON EARNINGS 

(Dollars) 



Quartet* 


District of Columbia 




Florida 




SJSA 


IJSA 


IJSA+ 


SJSA 


IJSA 


IJSA+ 


1 


30 


22 


22 


53 


-48 


-24 


2 


172" 


102 


147" 


-4 


-6 


20 


3 


152*" 


111 


176" 


-53 


-18 


14 


4 


28 1 ♦♦♦ 


161" 


83 


-2 


122 


50 


5 


280*" 


191" 


180" 


-92 


-36 


.. -12 


6 


241" 


183" 


106 


-66 


-36 


5 


7 


177* 


96 


-23 


-57 


-5 


63 


8 


263" 


.. 129 


38 


-98. 


-41 


-20 


9 


185* 


76 


10 


-98 


-41 


-49 


10 


224" 


100 


50 


-23 


-30 


-44 


11 








-33 


6 


14 


12 








-121 


50 


9 



“Full calendar quarters following initial UI claim. 

♦Statistically significant at the 90 percent level in a one-tailed test. 
♦♦Statistically significant at the 95 percent level in a one-tailed test. 
♦♦♦Statistically significant at the 99 percent level in a one-tailed test. 



Impacts on Job Search 

The JSA demonstration encouraged more aggressive job search efforts among treatment group 
members. In both D.C. and Florida, each of the JSA treatments led claimants to contact more employers 
per week in their job search, as shown in Table 3. For example, SJSA treatment generated 1.6 and 1.4 
additional contacts per week in D.C. and Florida, respectively. The LISA and IJSA+ treatments also 
increased the number of employers contacted as well as the hours spent searching for work. 
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TABLE 3 



ESTIMATED IMPACTS OF THE JSA TREATMENTS 
ON JOB SEARCH 





District of Columbia 




Florida 




Outcome 


SJSA 


IJSA 


IJSA+ 


SJSA 


IJSA 


IJSA+ 


Employers Contacted Per Week 


1.6* 


1.9* 


3.0* 


1.4** 


1.5* 


2.1” 


Hours of Search Per Week 


0.2 


0.6 


0.9 


0.4 


0.7 


1.7” 


Percent Receiving a Job Referral 
the Job Service 


from 8.7*** 


2.9 


8.7” 


3.4* 


3.8 


10.3”* 



’Statistically significant at the 90 percent level in a one-tailed test. 
’’Statistically significant at the 95 percent level in a one-tailed test. 
’’’Statistically significant at the 99 percent level in a one-tailed test. 



Another expected impact of the demonstration was to increase claimants’ use of the state Job Service 
(JS) to assist in their job search. All of the JSA treatments increased contact with the JS as designed, and 
they also tended to increase the probability that claimants received job referrals from the JS, as shown in 
Table 3. However, we found no evidence of impacts on the likelihood of obtaining job offers through JS 
referrals. Thus, it appears that the JSA demonstration was successful in getting claimants to use the JS, but 
that it was less successful in matching claimants to job offers generated through the JS. Regardless, the 
expanded use of JS may have helped claimants find jobs on their own. 



COST-EFFECTIVENESS OF THE JSA TREATMENTS 



Table 4 shows estimated costs per claimant, benefits per claimant, and the implied rate of return for 
each treatment from the perspectives of DOL, the government as a whole, and society as a whole. The 
cost estimates in the first panel of the table reveal that, as expected, the individualized JSA treatments were 
less costly than the structured treatment. The costs are constant across the three perspectives since all 
demonstration costs were incurred by DOL, and DOL costs are also a subset of both total government 
costs and societal costs. 



All of the JSA treatments yielded benefits for DOL primarily due to decreased UI payments. Most 
of the treatments also yielded benefits for total government. For society as a whole, the benefits estimates 
diverge substantially between D.C. and Florida. In D.C., the treatments yielded substantial benefits 
primarily due to the significant earnings increases caused by the treatments. In contrast, two of the three 
Florida treatments yielded negative benefits because our estimates suggest that the treatments reduced 
earnings (although the estimates are not statistically significant). 




xxiv 



22 



TABLE 4 



ESTIMATED COSTS AND BENEFITS OF THE JSA TREATMENTS 





District of Columbia 


Florida 




Perspective 


SJSA 


IJSA 


IJSA+ 


SJSA IJSA 


IJSA+ 


iitf - : JkiS! /filly! 




ts (Dollars 


lipii^§«§p 

per Claimant);, 






Department of Labor 
Total Government 


286 

286 


199 

199 


216 

216 


241 97 

241 97 


103 

103 


Society 


286 


199 


216 


241 97 


103 




Benel 


w&t&qp 

tits (Dollar 






Department of Labor 
Total Government 


160 

717 


89 

416 


31 

254 


17 97 

-110 77 


47 

54 


Society 


2,647 


1,552 


1,060 


-763 -119 


43 




Society 826% 680% 391% -416% -222% -59% 



The final step in our analysis of the JSA treatments was to combine the cost and benefit estimates to 
evaluate the cost-effectiveness of the treatments. In the final panel of Table 4 we present the estimated rate 
of return on the resources invested in each treatment, which is equal to net benefits (benefits-costs) divided 
by costs. 

Our estimates imply that the JSA treatments were not cost-effective from the perspective of DOL. 
Table 4 shows that none of the treatments in either state generated a positive return on the resources 
invested by DOL-the estimated reductions in UI payments caused by the treatments were not large enough 
to fully compensate for the costs of the services. The best case scenario implied by our estimates is that 
DOL would break even on their investment in JSA. 

Although the JSA treatments were not generally cost-effective from DOL’ s perspective, they may have 
been cost-effective from a broader perspective. The D.C. treatments generated substantial returns from 
the perspectives of government and society as a whole. For example, the societal rate of return for the 
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SJSA treatment was 826 percent, which implies that one dollar invested in SJSA yielded S8.26 in benefits 
for society. In contrast, the same treatments in Florida failed to generate positive returns for either the 
government or society as a whole. We hesitate to draw strong conclusions on the returns to society based 
on the Florida findings, since these estimates are sensitive to the earnings impacts, which are imprecisely 
estimated. In Florida, for example, the negative returns are driven partly by the finding that the treatments 
negatively affected earnings, but these negative estimates are statistically indistinguishable from zero. 



IMPLICATIONS FOR WORKER PROFILING AND RE-EMPLOYMENT SERVICES 

The Unemployment Compensation Amendments of 1 993 required states to develop Worker Profiling 
and Re-employment Services (WPRS) systems to identify UI claimants who might benefit from re- 
employment services and then refer them to re-employment services. These amendments directed all states 
to build their own statewide job search assistance systems. For WPRS, states are required to use the same 
two-step approach used in the demonstration to identify claimants to be referred to services. In most 
states, service referral in WPRS is similar to the USA treatment in the demonstration-each claimant is 
required to meet one-on-one with a counselor to develop an individual service plan and assess the 
claimant’s interests and abilities (Dickinson et al. 1999). Most states in WPRS also require at least some 
claimants to participate in individualized services beyond the standard mandatory services. However, as 
in IJSA, the percentage of claimants in any state actually required to participate in additional individualized 
WPRS services may be fairly low. 

The demonstration findings suggest that the typical WPRS service approach, which does not 
automatically require claimants to participate in services beyond orientation and assessment, is unlikely to 
generate widespread participation in other group services such as testing or job search workshops. To 
generate widespread participation, the states probably need to mandate these services. Findings from the 
WPRS evaluation presented in Dickinson et al. (1999) are largely consistent with this argument. Among 
the five states with valid data on service participation, the two states that explicitly required claimants to 
participate in a job search workshop as part of their WPRS requirements (New Jersey and Maine) 
generated fairly high workshop participation rates— about 40 percent or more. The other three states 
(Connecticut, Illinois, and South Carolina), which did not have explicit workshop requirements, generated 
much lower participation rates. Hence, it appears that in the early days of WPRS, substantial participation 
in many services was only achieved through explicit requirements that were backed up by the threat of 
benefit denials. 



Recommendation: If states want to expand services received by claimants through WPRS, 
states should make particular services mandatory for all claimants referred to WPRS, or at least 
encourage local offices to be aggressive in using individual service plans to set and enforce service 
requirements. 

Findings from the demonstration also suggest that coordination under WPRS between UI/JS and local 
agencies authorized to provide training under the Workforce Investment Act (WIA) may be difficult In 
both of the JSA demonstration states, as explained above, demonstration staff had some difficulty in 
working with EDWAA staff and getting claimants into EDWAA training quickly. This is consistent with 
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early observations of the WPRS systems presented in Hawkins et al. (1995), which reports that in many 
of the subject states, EDWAA played little or no role in WPRS. The researchers argue that improved 
linkages between EDWAA and the local UI and JS agencies involved in WPRS would allow the agencies 
to take better advantage of EDWAA expertise in serving dislocated workers with diverse needs. 
Coordination between UI/JS and EDWAA may have improved over time. Based on responses to a 1997 
survey, Dickinson et al. (1999) report that in 50 percent of states, EDWAA was substantially involved in 
at least one major WPRS task. Furthermore, EDWAA has now been replaced by WIA. The WIA 
requirement that local areas establish One-Stop Career Centers, which bring multiple agencies together in 
a single location to serve all clients, should contribute to improved coordination between UI/JS and the 
WIA agencies. 

Recommendation: DOL should continue to develop new tools, in addition to the One-Stop 
Career Centers, to encourage coordination of UI/JS and WIA and increase the exposure of 
WPRS claimants to WIA services. 

WPRS participation requirements are likely to increase UI nonmonetary benefit determinations and 
denials. Some of the increase will be due to direct enforcement of the WPRS requirements. But much of 
the increase will be due to more strict enforcement of traditional UT eligibility requirements. This kind of 
enforcement will be possible because of the additional information that local offices collect from claimants 
to track WPRS activities. Dickinson et al. (1999) confirm that WPRS increased nonmonetary benefit 
determinations and denials in most of the states that they examined. 

The JSA demonstration findings suggest that WPRS generates modest reductions in UI receipt. 
According to our estimates, the USA treatments, which most resembled typical WPRS services, reduced 
UI receipt by about half a week. Estimates from the WPRS evaluation reported in Dickinson et al. (1999) 
confirm that WPRS has an impact on UI receipt. WPRS reduced UI receipt in four of the six states 
investigated by Dickinson et al., with estimated reductions in the four states ranging from one-quarter of 
a week to one full week of benefits. 

Implications of the JSA demonstration findings for the impacts of WPRS on employment and earnings 
are more mixed. The USA treatments increased earnings in some quarters in D.C., but we found no clear 
evidence that the treatments increased earnings at all in Florida. Dickinson et al. also found no clear 
evidence that si mil ar services in WPRS increased employment or earnings, even in the states where UI 
receipt was significantly reduced. 

Finally, our findings provide little evidence that moving WPRS to a more structured model would be 
cost-effective. While in D.C. the rate of return on investment in SJSA was somewhat higher than on 
investment in USA, in Florida we found just the opposite. Furthermore, these comparisons are very 
sensitive to the earnings impacts, which are estimated imprecisely. 

Recommendation: Structured services do not necessarily maximize cost-effectiveness. States 
should use structured services only if their primary objective in WPRS is to expand service 
participation. 
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I. INTRODUCTION 



Increasing concern about dislocated workers has generated interest in finding effective and 
efficient methods for assisting these workers. The State Employment Security Agency (SESA) 
system, which is responsible for providing cash benefits and other re-employment assistance for 
unemployed workers in general, provides a setting in which dislocated workers can be identified and 
served early in their unemployment spells. SESA encompasses two agencies--the Unemployment 
Insurance (UI) system and the state Job Service (JS)~ that play key roles in this process. Workers 
who lose their jobs through no fault of their own and are covered by Unemployment Insurance (UI) 
can file, a claim for weekly UI cash benefits immediately after separation from their job. Following 
their initial claim, these workers, or claimants, are referred to the JS, which offers job placement 
assistance and other services to help claimants find new jobs. 

The Job Search Assistance (JSA) demonstration was designed to take advantage of the early 
contact unemployed workers have with the UI system to identify permanently separated workers who 
were likely to face lengthy UI spells and to help them find new jobs. Early intervention was 
emphasized in an effort to maximize the potential reduction in average UI spells. The demonstration 
was also designed to test alternative strategies for reducing UI spells, with the strategies varying in 
the degree to which services were customized for each worker. 

All of the service strategies tested in the JSA demonstration were designed to coordinate the 
efforts of the UI system, the state Job Service, and the local and state agencies providing services 
under the Economic Dislocation and Worker Adjustment Act (EDWAA). EDWAA services, which 
are intended specifically for dislocated workers, include classroom and on-the-job training, job 
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search assistance, and other reemployment services. By combining the efforts of these agencies, the 
demonstration was designed to offer a variety of services and address the claimants’ varying needs. 

A JSA demonstration in New Jersey during 1986 and 1987 indicated that the UI system could 
be used to identify dislocated workers and that re-employment services targeted at these workers 
could successfully reduce unemployment spells (Corson et al. 1989). After the success of the New 
Jersey demonstration, the Emergency Unemployment Compensation Act of 1991 (Public Law 102- 
164) authorized demonstrations to test this concept further in two additional states, Florida and the 
District of Columbia (D.C.). 1 By relying on an experimental design, the evaluation is able to 
measure how successful these services are at improving the employment outcomes of dislocated 
workers and reducing the burden on the UI system. 

This report examines the participation of claimants in the Florida and D.C. demonstration 
projects and the impact of the demonstrations on claimant outcomes over approximately two years 
following each claimant’s initial UI claim. This introductory chapter offers background about the 
issues that prompted the JSA demonstration and its evaluation, as well as a brief review of findings 
from the New Jersey demonstration. The chapter then describes the design parameters authorizing 
die JSA legislation and discusses how more recent legislation affected those parameters. Next we 
provide an overview of the demonstration design. The final section of the chapter explains how this 
report is organized. 



'For convenience, we refer to the District of Columbia as a state. 
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A. POLICY CONTEXT 



The JSA demonstration grew out of an effort to more effectively serve permanently laid-off 
workers. This section describes these workers and the various initiatives that preceded the JSA 
demonstration. 

1. Worker Dislocation and UI 

The re-employment problems of permanently laid-off workers have received national attention 
since the 1980s. Such workers have been called “dislocated” or “displaced.” After a layoff, many 
face long spells of unemployment and reduced wages when they are employed again. 

Since 1984, the Bureau of Labor Statistics (BLS) of the U.S. Department of Labor (DOL) has 
identified and tracked changes in worker dislocation through biannual supplements to the Current 
Population Survey (CPS). In this survey, workers who report “having lost or left a job because of 
a plant closing, an employer going out of business, a layoff from which they were not recalled, or 
other similar reason” are classified as dislocated. The 1994 survey showed that about 5.5 million 
workers were dislocated in 1991 to 1992. About half of this group, 2.8 million, had been employed 
in their jobs for three or more years (Gardner 1995). Displacements declined somewhat over the 
early 1990s. Data from the most recent CPS supplement demonstrate that in 1995 to 1996, 2.2 
million workers were dislocated from jobs they held for three or more years (Hippie 1999). 

In an analysis of these data on dislocated workers, the Congressional Budget Office (CBO 1993) 
found that about 2 million workers were dislocated each year during the 1980s. Although the 
numbers were higher than average during the recession of the early 1980s, substantial numbers were 
dislocated in all years, including those in which the unemployment rate was relatively low. The 
CBO study also found that workers in goods-producing industries-agriculture, mining, construction, 
and manufacturing— and in blue-collar occupations were at greater risk of dislocation than workers 
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in service-producing industries or white-collar occupations. However, many dislocated workers had 
been employed in service-producing industries and white-collar occupations as well. Moreover, 
differences in the risk of dislocation for these groups narrowed during the 1980s, a trend that 
continued in the early to mid-1990s (Gardner 1995 and Hippie 1999). 

The CBO study also showed that many dislocated workers had long spells of unemployment and 
lower wages after re-employment. One to three years after losing their jobs, half of them were either 
not working or had new jobs with weekly earnings less than 80 percent of their prior earnings. The 
workers with the largest losses had the least education, were the oldest, and had the longest tenure 
with the previous employer. Furthermore, dislocated workers who held a job at the time of the 
survey had had relatively long jobless spells--the average duration was just under 20 weeks. 

Other recent studies, which are summarized in Kletzer (1998), consistently demonstrate that a 
substantial earnings loss is associated with dislocation. Based on data from the Displaced Worker 
Surveys, Farber (1997) found that for workers dislocated between 1981 and 1995, real weekly 
postdislocation earnings were 13 percent lower than predislocation earnings. Jacobson, LaLonde 
and Sullivan (1993) used data from Pennsylvania to estimate that earnings of dislocated workers four 
years after separation were nearly $2,000 per quarter less than their nonseparated counterparts. 
Stevens (1997) also found large and persistent earnings losses due to dislocation based on a sample 
drawn from the Panel Study of Income Dynamics. One year after dislocation, earnings were 15 
percent lower for dislocated workers than for a comparison group of nondislocated workers. 

The UI system may be a convenient source for finding dislocated workers. First, many dislocated 
workers enter the UI system. For instance, the CBO study found that 70 percent of dislocated 
workers who were jobless for at least five weeks reported receiving UI benefits. More than half of 
these UI recipients were unemployed long enough to exhaust their benefits. More recently, in 1995 



to 1996, about one-half of dislocated workers reported receiving UI benefits after their jobs loss. 
The lower rate of UI receipt in 1995 to 1996 probably reflects the greater availability of jobs during 
that period, which enabled dislocated workers to find jobs very soon after, or even immediately 
following, their job loss. 

Another indication that the UI system is a useful tool for identifying dislocated workers is that 
many UI recipients can be classified as dislocated because they are permanently separated from their 
previous employers. Data from a study of UI recipients (Corson and Dynarski 1990) show that more 
than half of the UI population had no expectations of being recalled to their previous employer at 
the time they entered the UI system. Furthermore, about 36 percent of UI recipients in this study 
could be characterized as dislocated according to the BLS definition, which includes workers who 
lost their jobs because their plants closed, their employer went out of business, or their employer laid 
them off and they were not recalled. 

Dislocated workers who enter the UI system, like dislocated workers in general, have longer- 
than-average spells of unemployment and a greater likelihood of wage reductions than other 
claimants. Corson and Dynarski (1990) used their sample of UI claimants from 1988 to compare 
employment and UI benefit outcomes of dislocated and nondislocated workers. They found that 
dislocated workers, particularly those with substantial job tenure, had lower re-employment rates, 
longer spells of unemployment, higher UI exhaustion rates, and a lower ratio of post-UI to pre-UI 
weekly wages than did other claimants. For example, only 8 1 percent of the dislocated workers with 
three or more years of job tenure had become re-employed during the first 20 months after their 
initial claim; 92 percent of the nondislocated workers had become re-employed. 

These findings about re-employment problems suggest that dislocated workers may benefit from 
re-employment services. Moreover, since most dislocated workers who suffer long-term 
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unemployment enter the UI system, the UI system seems an effective mechanism to identify 
dislocated workers and direct services to them. 

2. The New Jersey UI Re-employment Demonstration 

The New Jersey UI Re-employment Demonstration was conducted in 1986 to 1987 to “examine 
whether the UI system could be used to identify workers early in their unemployment spells and to 
provide them with alternative, early intervention services to accelerate their return to work" (Corson 
et al. 1989). Overall, 8,675 UI claimants were assigned randomly to one of three treatments (job 
search assistance only, job search assistance combined with training or relocation assistance, and job 
search assistance combined with a cash bonus for early re-employment). The groups were compared 
with a randomly selected control group of 2, 385 claimants who received only regular services. 

Demonstration services were targeted at dislocated workers through a series of eligibility screens 
that excluded workers who (1) did not receive a UI first payment within five weeks after their initial 
claim, (2) were collecting partial UI benefits, (3) were younger than 25, (4) had fewer than three 
years of employment on their last job, (5) had a specific recall date from their employer, or (6) were 
usually hired through union hiring hall arrangements. Together these screens excluded 
approximately 73 percent of workers who received a first payment from UI during the sample period. 
Each treatment in the New Jersey demonstration had a statistically significant effect on reducing the 
collection of UI benefits and raising subsequent employment and earnings (Corson et al. 1989). The 
benefits of the treatments exceeded their total costs for society and the individuals involved. From 
the perspective of government alone, however, only the job search and ie-employment bonus 
treatments were entirely beneficial. No clear evidence emerged that providing training or relocation 
assistance in addition to job search assistance led to cost-effective gains. 
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3. Legislative and Programmatic Environment 

The Emergency Unemployment Compensation Act of 1991 authorized DOL to conduct the JSA 
demonstration to test the feasibility of implementing job search assistance programs and measure 
their effectiveness in promoting rapid re-employment of claimants. The legislation specified three 
eligibility criteria. To be eligible, UI claimants (1) had to have had job tenure at their last employer 
of at least 126 weeks at $30 or more per week during the three years ending on the last day of the 
base period, (2) had to have had no definite date for recall to former employment, and (3) could not 
be seeking work through a union hiring hall or similar arrangement. 

The legislation specified that the demonstration should test a basic job search assistance package 
similar to the one tested in New Jersey and test alternative treatments building on the basic treatment. 
The legislation also specified that an experimental design with random assignment of eligible 
claimants to a treatment group or a control group not receiving demonstration services be used to 
measure the effectiveness of the treatments in promoting re-employment. Claimants assigned to a 
treatment group were required to participate in basic treatment services in order to continue to 
receive UI benefits. 

Legislation enacted before implementation of the JSA demonstration affected this design. 
Specifically, the November 1993 Unemployment Compensation Amendments (Public Law 103-152) 
required states to implement a system to identify or “profile” claimants who are likely to exhaust 
regular compensation and to need re-employment services. These claimants were to be referred to 
services designed to help them become re-employed more quickly. Their participation in these 
services was to be mandatory. The legislation further directed states to implement these new Worker 
Profiling and Re-employment Service (WPRS) systems within a year, which meant they were in 
place before implementation of the demonstration. 
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These new requirements potentially affected the demonstration in two ways. First, the eligibility 
conditions specified in the legislation authorizing the JSA demonstration differed from the 
conditions specified for worker profiling. Therefore, it was likely that different groups of eligible 
individuals would be identified by the two systems. Because it did not make sense for states to 
implement two sets of eligibility conditions and procedures— one for the demonstration and one for 
profiling— congressional staff and DOL agreed to allow states to use the eligibility conditions and 
procedures selected for profiling for the demonstration as well. Moreover, while DOL gave states 
latitude in choosing eligibility conditions for profiling, both demonstration states agreed to use 
similar profiling models, which promoted consistency in the research design across the two states. 

Second, implementation of WPRS systems could affect the service environment of the control 
group for the JSA demonstration and affect the impacts measured in the demonstration. If the 
demonstration states provided an extensive set of re-employment services to the claimants identified 
through profiling, it would be difficult to measure demonstration treatment impacts because one or 
more of the treatments would probably be similar to the services environment faced by the control 
group. 

However, the UI claimant population in the District of Columbia (D.C.) was too small to support 
implementation of the demonstration and a WPRS system, and the decision was made to implement 
the demonstration initially and then implement a WPRS system once the demonstration was 
completed . 2 Since the JSA demonstration was essentially a test of the WPRS concept, the transition 
from the demonstration to WPRS would be straightforward. Florida made a similar decision by 
deciding not to implement its WPRS system in the local offices chosen for the demonstration until 

2 Because the demonstration eligibility procedures and services were similar to those of a WPRS 
system and because the demonstration encompassed all offices in D.C., implementation of the 
demonstration in D.C. satisfied the federal requirement to implement a WPRS system. 
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the demonstration was completed. The demonstration was considered a temporary substitute for 
WPRS. 

B. OVERVIEW OF THE DEMONSTRATION DESIGN 

The JSA demonstration implemented three packages of re-employment services. Eligible 
claimants were identified early in their claim periods, using a profiling model. These claimants were 
then assigned to the control group or to one of three treatments: (1) structured job search assistance, 
(2) individualized job search assistance, or (3) individualized job search assistance with training. 

D.C. and Florida were chosen for implementing the demonstration based on plans submitted to 
DOL. In D.C., 8,071 eligible claimants were selected from the entire new UI claimant population 
during a one-year intake period (June 1995 to June 1996). In Florida, 12,042 eligible claimants were 
selected from new claimants in 10 local Jobs and Benefits offices over a one-year intake period 
(March 1995 to March 1996). 3 

1. Eligibility Criteria for the Demonstration 

A profiling model approach developed by DOL was used to identify claimants eligible for the 
demonstration. This model used a two-step process, described below, to identify the claimants 
expected to experience long spells of unemployment. 

In the first step, claimants were screened out of the demonstration if they had characteristics that 
suggested the demonstration services were clearly inappropriate for them. The intent was to exclude 
claimants who had permanent ties to their previous employer or some other reason to not actively 




3 Shortly before the demonstration, Florida merged UI field claims operations with Job Service 
offices. The new entity, Jobs and Benefits, is responsible for both UI claims and Job Service 
programs, and the local offices are now called Jobs and Benefits (J&B) offices. To simplify our 
presentation, we sometimes refer to local J&B offices as UI offices and local J&B staff as UI or Job 
Service staff when we are discussing D.C. and Florida together. 
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search for work, claimants who had already been unemployed for a long time (which would nullify 
the early intervention goal), and claimants who faced severe obstacles to participating in the 
demonstration. The specific screens used in each state are discussed in Chapter III. 

Then, state-specific regression models were used to predict the probability of exhaustion of 
benefits, and claimants with a high probability of exhaustion were chosen as the demonstration- 
eligible population. These regression models, which were based on a national model developed by 
DOL, used five variables to predict exhaustion: (1) the claimant's pre-UI industry, (2) the claimant’s 
pre-UI occupation, (3) the claimant’s education, (4) the claimant’s years of tenure on the pre-UI job, 
and (5) the local unemployment rate. These models are described in greater detail in Appendix B. 

2. Re-employment Service Packages 

Eligible claimants were identified as soon as the first UI payment was made. They were then 
randomly assigned to one of three treatment groups that received demonstration services or a control 
group that received regular services. The three treatment groups, as shown in Figure 1.1, were 
offered the following services: 

1 . Structured Job Search Assistance (SJSA). This treatment replicated the basic job 
search assistance treatment tested in New Jersey. Claimants were sent a letter during the 
fourth week of unemployment telling them to report to a Job Service orientation session. 4 
Claimants reported for orientation two to three weeks later, approximately during the 
sixth or seventh week of unemployment 




4 This description of the timing of services assumes claimants applied for Ul benefits as soon as they 
were laid off. 
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DESIGN OF TREATMENTS, BEGINNING WITH ORIENTATION 
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At the orientation, claimants were told about the re-employment services available to 
them and specifically about demonstration services. They were tested the same week and 
scheduled for an approximately 15-hour job search workshop the following week. After 
the workshop, they were scheduled for a one-on-one assessment/counseling interview to 
discuss their re-employment plans. Attendance at this initial set of services was 
mandatory, unless the claimant was explicitly excused from services. Claimants who 
failed to participate could lose some of their UI benefits. In subsequent chapters, we 
describe the participation rates in each of the services and the enforcement of the 
participation requirements. 

Individuals who continued collecting UI benefits were to have a minimum of two 
additional contacts with staff. These contacts could take any form. In addition, each 
office established a job search resource center for demonstration participants, and 
participants could receive any other services offered by the Job Service. However, these 
services were not mandatory. 

2. Individualized Job Search Assistance (IJSA). This treatment was similar to the job 
search assistance treatment, except that a decision was made on an individual basis about 
the services a claimant should receive. Eligible claimants were sent a letter telling them 
to report for a group orientation session held during approximately their sixth or seventh 
week of unemployment. At that session, they were given an overview of the services 
available to them and scheduled for an individual assessment interview later that week 
or the next week. An individual service plan was developed during the assessment 
interview. This plan varied across individuals, but the specified services— such as testing, 
the job search workshop, or additional assessment/counseling interviews— were 
mandatory. Additional ongoing contacts could also be required. Claimants in this 
treatment could also receive any other services, such as placement assistance, from the 
Job Service, but these services were not mandatory. In subsequent chapters we describe 
the service participation and the use of the service plan for the typical claimant assigned 
to this treatment. 

3. Individualized Job Search Assistance with Training (IJSA+). This treatment was 
identical to the second treatment, but included a special effort to enroll interested 
claimants in training. In all treatments, EDWAA training was described during the 
orientation session. Claimants who expressed interest in training were referred to 
EDWAA. However, in this treatment, the discussion of training opportunities during 
orientation was more extensive and, if possible, made by an EDWAA staff member. 

In addition, the possibility of training was explicitly discussed during the individual 
assessment interview. Any claimant who expressed interest was scheduled to talk to an 
EDWAA staff member. If possible, this discussion took place in the Job Service office 
immediately following the assessment interview. 
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This “one-stop” approach to service delivery was to be facilitated by having assessment 
interviews on days when an EDWAA staff member could be at the Job Service office. 

To ensure that training was available, the states participating in the demonstration were 
asked to designate a portion of their EDWAA funds to provide training to members of 
this group. 

C. RESEARCH METHODOLOGY AND DATA 

A central feature of the JSA demonstration is the random assignment of demonstration-eligible 
UI claimants to a control group or one of the three treatment groups. Random assignment ensures 
that the treatment and control groups exhibit similar characteristics and each is representative of the 
target population. Based on this random assignment design, any differences in outcomes between 
each of the three treatment groups and the control group are directly attributable to the JSA service 
packages and the differences therefore provide unbiased estimates of the treatment net impacts. 
Given this, we estimate the net impacts of the three treatments separately by comparing the average 
outcome for each treatment group with the average outcome for the control group . 5 In a couple of 
cases we extend the research methodology beyond the simple treatment-control differences in order 
to address specific research questions that are beyond the scope of the treatment-control design. The 
cases are highlighted in the text of the report, where we also describe the methodology that is used. 

The claimant-level data used in the impact analysis come from four sources: the JSA 
demonstration participant tracking system ( PTS), UI program data, UI wage records, and a one-year 
follow-up survey of a subset of demonstration claimants. We also use data from site visits and 
published reports to describe the demonstration environments (Chapter II), estimate the program 



5 We also tested whether using regression models to adjust for claimant characteristics would 
generate different impact estimates and found that the adjusted estimates were nearly identical to the 
unadjusted treatment-control differences. We decided therefore to report only the unadjusted 
treatment-control differences. 
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costs (Chapter IV), and provide other contextual data to interpret the findings from the analysis of 
claimant-level data. 



ERIC 



1. JSA Demonstration PTS 

We describe the participation in and timing of JSA services (Chapter III) based on data drawn 
from the JSA demonstration PTS. The PTS was a computer-based system that was used to assist 
in the operation and monitoring of the demonstration. It was also used to identify and select the 
sample, to assign eligible claimants randomly to the treatment and control groups, and to send 
notices to claimants to report to services. With respect to services, the PTS contained data on service 
assignment, attendance, and timing. 

2. UI Program Data 

We estimate the impacts of the JSA treatments on UI outcomes (Chapters V and VI) based on 
UI program data collected from each state. At the end of the JSA demonstration operations, we 
received special data extracts from the UI mainframe system in each state. These extracts contained 
UI payment records that were used to create measures of UI benefit receipt for each claimant in the 
demonstration. These extracts also contained data on the UI base period earnings, the pre-UI 
employer, and UI determinations and denials. 

3. UI Wage Records 

Our estimates of the impacts of the JSA treatments on employment and earnings (Chapter VII) 
are based primarily on quarterly earnings data drawn from the state UI wage records. The advantage 
of this source of earnings data is that wage records can be easily obtained for most sample members. 
However, the wage records provide somewhat crude measure of post-UI earnings. Because the data 
are organized by quarter, they cannot be used to determine the point at which claimants become 
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reemployed after claiming UI. Furthermore, the wage records do not include earnings from self- 
employment, federal jobs, military services, or domestic or agricultural employment. 

Given these limitations, we collected additional detail on employment and earnings as part of the 
follow-up survey. But because of the broader coverage provided by UI wage records and the 
potential nonresponse bias associated with the follow-up survey data (see Appendix A), we use the 
wage records as our primary source of data on employment and earnings outcomes. 

4. Follow-Up Survey Data 

The follow-up survey was conducted approximately one year after enrollment. The survey 
provided data on the following: 

D Demographic information, such as household composition and education 

□ Characteristics of the prelayoff job 

D Perceptions of and attitudes towards the JSA demonstration services 

□ Participation in alternative job-search programs 

□ Postlayoff job search behavior, employment, and earnings 

□ The length of postlayoff unemployment 

□ Sources and amounts of non-UI income, including the labor force activities of spouses 
during the pre- and postlayoff periods 

Although we collected information on employment and earnings from the UI wage records, we 
also collected these data from the follow-up survey. The survey data on employment and earnings 
are useful both to potentially address some of the limitations of the wage records data and to provide 
detailed information on the characteristics of post-UI jobs. For this report, survey data were also 

used to construct the outcomes measures for the analyses of customer satisfaction with JSA services 
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(Chapter III) and the treatment impacts on job search activities and post-UI job characteristics 
(Chapters VIII and IX). The details associated with the follow up survey are described in Appendix 
A. 

D. CONTENT OF THE REPORT 

This report on the JSA demonstration examines the operation and impacts of the demonstration 
in D.C. and Florida. We first describe the economic and service environments in D.C. and Florida, 
as well as the UI population targeted by the demonstration in each state. These issues are addressed 
in Chapter II. The next two chapters address the operation of the demonstration-chapter III 
describes the services received by claimants and the timing of the services, while Chapter IV presents 
the estimated costs of providing the services. Chapters V, VI, VII, VIII, and IX present our estimates 
of the impact of the demonstration on various claimant outcomes— UI receipt, UI benefit eligibility, 
employment and earnings, job characteristics, and job search activities. Chapter X presents estimates 
of the cost-effectiveness of the treatments. Chapter XI summarizes our findings and discusses the 
implications of our findings for the new WPRS systems and for displaced worker policy in general. 
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II. THE DEMONSTRATION ENVIRONMENT AND 
ELIGIBLE POPULATION 



Economic conditions influence whether individuals can obtain employment and their decisions about 
when to accept employment. Understanding the characteristics of the eligible population and the prevailing 
economic conditions will help us assess whether the demonstration findings can be generalized for claimant 
populations outside the demonstration. In this chapter, we review the process used to select local sites for the 
demonstration and describe the economic environments in these sites and in D.C. and Florida more broadly 
during the demonstration. We also describe state and local factors related directly to the demonstration. First, 
we consider the characteristics of the claimants who were determined to be eligible for the demonstration in 
each state and site. Second, we describe the organization and staffing of local offices and agencies 
participating in the demonstration. Finally, we examine the approach each state took to monitor and enforce 
compliance with the participation requirements. 

General economic conditions in both states at the beginning of the demonstrations were somewhat less 
favorable than in the nation as a whole. However, both states had recent histories of high economic growth, 
and both, especially D.C., depend primarily on non-manufacturing industries with relatively high growth in 
recent years. 

The claimant population targeted by the demonstrations was somewhat older in Florida than in D.C. 
Florida also had both more high school dropouts and college graduates among its claimants than D.C. did. 
The Florida claimants were much more likely than the D.C. claimants to have held manufacturing jobs, while 
D.C. claimants were more likely to have held service or public administration jobs. In Florida, the 
characteristics of the eligible claimants varied greatly across sites. Compared with claimants nationwide, the 
D.C. and Florida demonstrations tended to include more women claimants, fewer young claimants, more 
African American and Hispanic claimants, and many fewer claimants from manufacturing. 
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A. SELECTION OF LOCAL OFFICES 



Our selection of local offices in both demonstration states was designed to maximize the generalizability 
of the demonstration findings to the entire state. In D.C., achieving generalizability to the entire state was 
relatively straightforward because of its small size. The demonstration served claimants from every UI office 
and those applying in suburban offices for benefits from D.C. 1 The demonstration participants were selected 
randomly from those claimants who were determined to be eligible based on the profiling and selection 
procedures that we describe in Appendix B. 

In Florida, we did not include every local Jobs and Benefits (J&B) office in the demonstration because 
providing services in every office in the state would have entailed large operational burden and costs. A more 
cost-effective approach was to select a random sample of local offices and then select a random sample of 
claimants from each of the selected offices. Selecting offices and claimants randomly ensures that the 
demonstration-eligible claimants are representative of the Florida UI population and that the results of the 
demonstration can be applied to the state population. 

A related design objective was to assign each potential eligible claimant an equal probability of selection. 
This procedure yields the most efficient sample design for generalizing to the entire eligible population if no 
particular subgroup of the eligible population is of greater interest than another. In D.C., where the entire UI 
population was covered by the demonstration, we selected claimants at random from the eligible UI population 
without regard for where they filed their initial UI claim. In Florida, the objective of equal probability of 




‘Because Washington, D.C. is part of a large metropolitan area, it allows claimants filing for 
benefits to file their initial claim in one of the suburban Virginia or Maryland offices. The D.C. UI 
system treats these claimants the same as if they had filed their claim in a D.C. office, so we included 
them in the demonstration. 
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selection implied that local offices should be selected with the probability of selection proportional to the size 
of the eligible population and that equal numbers of eligible claimants be offered services in each office. 

A final objective in selecting local offices in Florida was to achieve a broad representation of local offices 
and a broad representation of eligible claimants. To help achieve this objective, the local offices were stratified 
geographically and selected randomly within the geographical strata. 

Two other issues were taken into account in selecting Florida offices. First, some offices served a 
population not suitable for the demonstration. A few offices served primarily agricultural workers on seasonal 
layoffs, so we excluded these because most claimants would not be looking for a new job. The state identified 
three of the 49 local offices according to this criterion. Second, some offices were considered too small to 
provide an adequate sample of eligible claimants for the demonstration. We excluded 16 additional offices 
that had average claimant populations below 3,200 in 1992 and 1993. We also excluded the Tampa office 
because it was experimenting with local service initiatives, including one-stop shopping, and it therefore did 
not provide an accurate representation of typical Florida local J&B operations. In addition, we wanted to avoid 
the potential for confusing local staff by asking them to implement another service initiative. Although the 
number of offices excluded in Florida (20) was high, the excluded offices represented only 16 percent of the 
claimant population. 

We selected 10 local Florida offices based on this process: Pensacola, St. Augustine, Orlando, Lakeland, 
Clearwater, Fort Pierce, Fort Lauderdale, Davie, Hialeah, and Miami (see Figure II. 1). 

B. CHARACTERISTICS OF DEMONSTRATION SITES 

The two areas operating the JSA demonstration are obviously quite different. Florida is one of the most 
populous states in the United States and has a recent history of extraordinarily high economic and population 
growth. The state encompasses several large cities, as well as many smaller cities and 
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large rural areas. In contrast, Washington, D.C., is a single city, and the federal government is the largest 
employer. Although metropolitan Washington has grown rapidly in recent decades, the city itself has lost 
population. Both D.C. and Florida had somewhat mixed economic conditions during the period of the 
demonstration, as we describe in this section. 

1. District of Columbia 

The city of Washington D.C. has a population of 570,000 residents; the Washington Metropolitan 
Statistical Area (MSA), includes nearly 4.5 million residents from the city and its surrounding suburbs. Both 
Washington, D.C., and the Washington MSA provide useful information about the economy and labor markets 
UI claimants face. Many of the claimants either already live outside D.C. or have easy access to employers 
located in the suburbs, so it is sensible to use the metropolitan area as representative of the local economy. 
Other claimants may have more limited access to the Washington suburbs, so D.C. may provide a more 
relevant definition of the local market for those claimants. 

The economic statistics suggest that economic conditions were relatively favorable in the Washington 
metropolitan area, but not as favorable within D.C. For example, the metropolitan area had an unemployment 
rate in 1994 of only 4 percent, which was considerably lower than the national rate of 6 percent (Table II. 1). 
In contrast, D.C. had an unemployment rate of 8.2 percent. Similarly, between 1990 and 1994 employment 
grew by 5.9 percent in metropolitan Washington, compared with a 6.1 percent decline in D.C. Metropolitan 
Washington has little employment in manufacturing-only 2 percent within D.C. and 4 percent in the 
metropolitan area. Not surprisingly, the proportion of government employment is high. However, during the 
demonstration, government employment was declining. For D.C., the decline in government employment 
translated into an overall employment decline during this period. For the overall metropolitan area, 
employment growth 
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TABLE II. 1 

CHARACTERISTICS OF THE D.C. DEMONSTRATION SITE 





District of Columbia 


Washington 
Primary MSA 


Population (1994, in thousands) 


570 


4,474 


Population Growth (1990-94, percent) 


-6.1 


5.9 


Unemployment Rate (1994, percent) 


8.2 


4.0 


Percent Manufacturing Employment (1995) 


2.1 


4.0 


Percent Government Employment (1995) 


38.6 


25.3 


Percent Change in Employment (October 1994 
to October 1995) 






Total 


-2.6 


0.5 


Manufacturing 


0.8 


2.5 


Trade 


0.2 


2.0 


Services 


-0.6 


2.7 


Government 


-6.1 


-2.9 


Per Capita Income (1993, dollars) 


29,500 


25,956° 



“Based on the Washington-Baltimore consolidated MSA. 

MSA = Metropolitan statistical area. 

SOURCE: District of Columbia Department of Employment Services, Area Labor Summary, 

December 1995; U.S. Department of Commerce, Bureau of the Census, Statistical Abstract 
of the United States, 1995. 
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in other industries compensated for the decline in government employment, so employment overall increased 
slightly. 

2. Florida 

Although Florida has enjoyed many years of rapid economic growth, the unemployment rate for Florida 
(6.6 percent) was higher than for the nation (6.1 percent) in 1994, when the demonstration began. For the 
period from April 1995 to April 1996, when most demonstration participants were searching for work, total 
employment in Florida grew by 3.2 percent. Strong state growth in service employment compensated for a 
slight decline in manufacturing employment over this period (Table 11.2). 

In selecting local offices in Florida for the demonstration, the offices were stratified geographically and 
then selected randomly from the strata with probability of selection proportional to size. The offices selected 
for the demonstration therefore were scattered throughout the state (see Figure II. 1), and many of the offices 
were located in relatively large population centers. 

Four of the 10 demonstration offices in Florida were located in the populous southeastern coastal area 
of the state near the city of Miami (see Figure II. 1). Two offices, Fort Lauderdale and Davie, were located 
in Broward County; two other offices, Miami and Hialeah, were located in neighboring Dade County. The 
populations of Broward and Dade counties, which are the largest in Florida, are concentrated in 
urban/suburban communities along the coast of the Atlantic Ocean. Population and employment growth 
have been relatively strong in Broward County in recent years. The 1994 unemployment rate in Broward 
County was nearly 6.5 percent, which was comparable to Florida as a whole (Table II.2). Between April 
1995 and April 1996, which covers most of the demonstration period, total employment in the Fort 
Lauderdale MSA grew by nearly 4.5 percent. Employment 
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CHARACTERISTICS OF THE FLORIDA DEMONSTRATION SITES 
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growth was especially strong in trade and services employment, which grew by more than 5 percent. The 1993 
per capita income in Broward County was $23,840, which was higher than for the state as a whole and ranked 
seventh among all Florida counties. 

Dade County includes the city of Miami and the corresponding suburban areas. Population and 
employment growth in Dade County have been slower than growth in Broward in recent years. Between 1990 
and 1994, the population of Dade County grew by less than 3 percent. The unemployment rate for the county 
was 8.0 percent, somewhat higher than the statewide rate of 6.6 percent. The proportion of county 
employment in manufacturing is relatively modest (7.8 percent), and this proportion declined between 1983 
and 1993, as manufacturing employment declined by nearly 10 percent Manufacturing employment continued 
to decline, at least in the Miami MSA, by 3 percent between April 1995 and April 1996. Employment growth 
over this period in Miami was strongest in services and construction. The per capita income in Dade County 
in 1993 was $19,266, which ranked 16th among the counties but was less than the statewide income figure 



($20,650). 

Two other demonstration sites were located in communities on the Atlantic Coast. St. Augustine and Fort 
Pierce are located in the least populous counties with demonstration offices, St. Johns County and St. Lucie 
County, respectively. Although the populations in these counties are small, they grew rapidly between 1990 
and 1994--by 13 percent in St. Johns and 11 percent in St. Lucie. The economic conditions in these two 
counties are very different. St. Johns is a relatively affluent county with a per capita income in 1993 of 
$24, 797-sixth among all counties in the state and first among the counties with demonstration sites. The 1994 
unemployment rate in St. Johns was 5.9 percent, which was below the average for the state and for the 
demonstration sites. Manufacturing employment, although a modest proportion (8.2 percent) of all 
employment, grew rapidly between 1983 and 1993. More recently, employment growth in St. Augustine was 
2.4 percent between April 1995 and April 1996, which was below the state average. The highest employment 
growth occurred in services. 
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Compared with St. Johns County, St. Lucie is less affluent and has encountered less favorable economic 
conditions and slower employment growth. St. Lucie’s per capita income in 1993 was $15,773, substantially 
below the state average and the lowest income level among the counties with demonstration sites. The 1994 
unemployment rate (13.4 percent) was about twice the state rate and the highest rate among the demonstration 
sites. This high unemployment rate may be an indication that job seekers in St. Lucie County faced substantial 
barriers to re-employment during the time of the demonstration. Despite the high county unemployment rate 
in 1994, the rate of employment growth in Fort Pierce between April 1995 and April 1996 was 2.4 percent, 
which was similar to the rate for St. Johns County and below the state rate. 

Three of the demonstration offices (Clearwater, Lakeland, and Orlando) are located in central Florida. 
Clearwater and Lakeland are located in Pinellas County and Polk County, respectively, which have had 
relatively slow population growth in recent years. Population in both of these counties grew by less than 3 
percent between 1990 and 1994, which is less than half the rate for the state. Total employment in the 
Clearwater and Lakeland MSAs expanded during the demonstration as strong growth in services and trade 
made up for the decline in manufacturing employment in each site. Despite these similar trends, the 1994 
unemployment rate in Polk County was a relatively high 8.3 percent, compared with a relatively low 5.2 
percent in Pinellas County. This difference in economic conditions is also reflected in the income levels of 
the two counties. Per capita income in Pinellas County in 1993 ($22,798) was higher than the state as a 
whole, while that in Polk County ($16,858) was lower. 

The other central Florida demonstration office, Orlando, is located in a county with strong population 
growth in recent years. The population of Orange County grew by 9.25 percent between 1990 and 1994. The 
county unemployment rate (5.8 percent) was a bit lower than that for the state as a whole. As in other Florida 
counties, employment in manufacturing represents a small proportion (6.7 percent) of all employment, but 
manufacturing employment in Orange County expanded by 20 percent between 1983 and 1993. Over the 
demonstration period, employment in Orlando grew by slightly more than 3 percent, matching the growth rate 
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statewide. Orange County is not one of the wealthiest counties in the demonstrati on— it ranks 15 th among 
Florida counties, with per capita income that was slightly lower in 1993 than that of the state as a whole. 

The remaining demonstration site, Pensacola, is the largest city in Escambia County in the Florida 
panhandle. In 1994, Escambia had an unemployment rate of 5.1 percent, which was the lowest of the 
demonstration sites. Only 6.6 percent of the employed residents in 1994 held jobs in manufacturing. 
Government employment represents a large proportion of the county workforce— second only to services in 
1994. The large government employment arises because of the location of large military bases in the 
Pensacola area. Manufacturing employment in Pensacola remained constant over the demonstration period, 
while service and construction employment expanded rapidly. The overall employment growth rate was 
comparable to the state as a whole. The income level in Escambia is relatively low-$ 16,899 in 1994, 
considerably lower than the statewide average. 

C. CHARACTERISTICS OF THE ELIGIBLE POPULATION 

Eligibility of the claimants for the demonstration was based on data collected from claimants when they 
filed their initial UI claim. We make comparisons between the eligible claimants in the two states, and we 
compare the claimants in the two states with a nationwide sample of UI claimants from 1988 (Corson and 
Dynarski 1990). The measures we present for the eligible population are based on the demonstration 
participants— those claimants assigned to one of the treatment groups or the control group. Since this group 
is a random sample of all eligible claimants, measures based on this group provide valid estimates of the 
corresponding measures for the eligible population. 

Differences between the eligible claimants in the demonstration states and the nationwide sample arise 
because of three factors. First, there are differences between claimants in D.C. and Florida and claimants 
nationwide. For example, relatively few claimants in D.C. or Florida come from manufacturing because of 
the limited manufacturing employment in either state. Second, the demonstration eligibility criteria also affect 




28 



56 



the degree to which the eligible claimants in the demonstrations differ from claimants nationwide. For 
example, the eligibility criteria screen out claimants who have an expected date of recall to their previous 
employer. Since recall is more prevalent in manufacturing than in other industries, the eligibility criteria 
contribute to D.C. and Florida demonstration eligible claimants being less likely to be from manufacturing than 
claimants nationwide. Finally, there may be differences between new UI claimants in 1988 and new claimants 
in 1995 and 1996. This factor probably plays a minor role compared with the other two factors in explaining 
the differences between the claimant samples. 

1. By State 

Eligible claimants in the D.C. and Florida demonstrations had very different characteristics. As 
shown in Table 11.3, the representation of women in the eligible populations in both states was higher than 
the representation of women in the nationally representative sample of all UI claimants from 1988. 2 
According to the 1988 sample, about 41 percent of all U.S. claimants were women, as shown in the final 
column of Table II.3, compared with 46 percent of eligible claimants in Florida and 55 percent of eligible 
claimants in D.C. This difference probably arises from the initial eligibility screens used in the JSA 
selection process. Some of the screens, such as the union hiring hall screen, may 
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2 The statistics presented in Table II.3 differ slightly from the corresponding statistics on sample 
characteristics presented in the JSA implementation report (Decker, Perez-Johnson, and Corson). The 
differences arise because in this report we have excluded claimants who entered the demonstration 
during the pilot phase. These claimants were included in the implementation report. 
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CHARACTERISTICS OF THE DEMONSTRATION ELIGIBLES 
(Percent) 
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mostly affect claimants in industries that are dominated by male workers. The screens would therefore tend 
to exclude male claimants from the demonstration at a higher rate than female claimants. 

The Florida claimants were older than D.C. claimants by an average of five years~43.1 years old in 
Florida compared with 38.0 years old in D.C. Florida had many more claimants in the oldest group. Over 5 
percent of the Florida claimants were 65 years or older, compared with less than 1 percent of D.C. claimants. 
The Florida claimants also tended to be older than the average UI claimant across the country. According to 
data from 1988, the average UI claimant nationwide was about 38 years old, considerably younger than the 
Florida claimants, but comparable to the D.C. claimants. The findings from the initial states to implement 
Worker Profiling and Re-employment Services systems show that similar profiling and screening methods 
used in those states also targeted claimants who were older on average than nontargeted claimants (Dickinson, 
Kreutzer, and Decker 1997). 

Racial minorities made up a substantial proportion of demonstration-eligible claimants in both D.C. and 
Florida. A large majority of D.C. claimants (83 percent) were African American. In Florida, the majority (62 
percent) of claimants were white and non-Hispanic, but a substantial proportion of claimants were African 
American (16 percent) or Hispanic (22 percent). In contrast, the percentages of African American and 
Hispanic claimants in the 1988 national sample were only 12 and 10 percent, respectively. 

Demonstration-eligible claimants in the two states were distributed very differently among education 
levels, although our comparisons are limited by the different education codes used in the two states (see 
footnotes in Table II.3). Florida had more claimants than D.C. at the ends of the education scale. The 
proportion of claimants who were high school dropouts was considerably higher in Florida (26 percent) than 
in D.C. (15 percent). But the rate of college completers was also higher in Florida, by about 50 percent, than 
in D.C. Compared with the nationwide claimants, Florida had a higher proportion of school dropouts and a 
similar proportion of college completers, while D.C. claimants had a lower proportion of dropouts and a lower 
proportion of college completers. 
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Demonstration-eligible claimants in the two states also had different pre-UI job experiences. Florida 
claimants were about 10 times more likely than D.C. claimants to have previously held a manufacturing job- 
less than 2 percent of D.C. claimants held jobs in manufacturing, compared with 15.7 percent in Florida. But 
both states had a relatively low proportion of eligible claimants from manufacturing when compared with the 
1988 national sample of UI claimants, where manufacturing workers made up nearly 40 percent of all 
claimants nationwide. As stated in the introduction to this section, this difference is attributable both to special 
characteristics of D.C. and Florida claimants in general and to the demonstration eligibility criteria. Other 
differences in industry between eligible claimants in D.C. and Florida were less stark than the manufacturing 
differences, but they were still important. D.C. claimants were less likely than Florida claimants to have had 
jobs in transportation and utilities, wholesale and retail trade, and finance, insurance, and real estate. On the 
other hand, D.C. claimants were more likely to have held jobs in services or public administration. 

Average UI entitlements and weekly benefit amounts were higher among D.C. demonstration-eligible 
claimants than their Florida counterparts. This was at least partly because the D.C. maximum weekly benefit 
amount ($347 until January 1996, when it increased to $359) was considerably higher than the Florida 
maximum weekly benefit amount ($250). D.C. claimants had higher entitlements than Florida claimants 
despite having lower average base period earnings. 

Finally, for each claimant we calculated, based on claimant characteristics, a predicted probability of 
benefit exhaustion that was used to target the demonstration services. 3 The predicted probability of UI benefit 




3 The predicted probability is based on a statistical model relating the probability of benefit 
exhaustion to claimant characteristics. See Appendix B for a detailed description of this model, which 
is used to target the demonstration services to claimants with the highest expected probabilities of 
exhaustion. 
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exhaustion was higher for D.C. claimants-7 1 . 1 percent compared with 51.3 percent for the Florida claimants. 4 
This difference is a reflection of the traditionally high rate of benefit exhaustion in D.C. In 1990, for example, 
53 percent of all D.C. claimants exhausted their benefits, compared with 42 percent of Florida- claimants. 

2. By Site in Florida 

Within Florida, the demonstration eligible population varied greatly across the sites. One apparent 
difference was the age of the claimants. Claimants in Fort Lauderdale, Davie, Hialeah, Miami, and Clearwater 
tended to be older than the claimants in the other sites. The average age of claimants in these five sites ranged 
from 44 to 47 years, compared with 39 years in Pensacola and 41 to 42 years in St. Augustine, Orlando, 
Lakeland, and Fort Pierce. 

The racial and ethnic background of claimants also varied widely. More than 90 percent of the claimants 
in Clearwater were white and non-Hispanic, compared with less than 20 percent of Miami claimants and less 
than 10 percent of Hialeah claimants. Sites with high proportions of African American claimants included Fort 
Pierce (25.7 percent), Miami (24.7 percent), Pensacola (24.1 percent), and Lakeland (19.2 percent). Sites with 
high proportions of Hispanic claimants included Hialeah (83.4 percent), Miami (58.1 percent), Davie (19.9 
percent), and Orlando (18.7 percent). 

Some of the Florida sites served claimant populations with limited educational backgrounds. In Hialeah 
and Miami, more than 40 percent of the claimants determined to be eligible for the demonstration had not 
completed high school. College completion was less than 5 percent among Hialeah and Fort Pierce claimants. 
Educational attainment was higher in other Florida sites. The high school dropout rate was lowest among 
claimants in Pensacola (14.2 percent) and Orlando (15.0 percent). College completion was more than 10 
percent among claimants in Clearwater, Orlando, Fort Lauderdale, and Davie. 




4 The predicted probabilities of exhaustion were calculated for each claimant as part of the 
demonstration eligibility determinatioa 
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Although the proportion of claimants from manufacturing in Florida was low overall, more than 20 
percent of the claimants in Lakeland and Hialeah were from manufacturing. In contrast, less than 10 percent 
of Pensacola claimants were from manufacturing. Pensacola had a much higher proportion of claimants from 
public administration (15 percent) than other Florida sites (less than 5 percent). This difference is probably 
because of the presence of major military bases near Pensacola, whose employees would be classified as being 
in public administration. Since the demonstration was operating during a time of military downsizing, a 
substantial proportion of the Pensacola claimants were former employees at the military bases. Fort Pierce 
and Lakeland were noteworthy for having relatively high proportions of agricultural and mining workers--10 
percent in Fort Pierce and 4 percent in Lakeland. No other site had more than 2 percent of claimants from 
agriculture and mining. 
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HI. PARTICIPATION, TIMING OF SERVICES, AND 
CUSTOMER SATISFACTION 



Understanding claimants’ participation in services is important for two reasons. First, data on 
participation in demonstration services and the timing of services provides important information 
on the extent to which the demonstrations were administered as designed. Unusually low 
participation, participation in services out of sequence, or delays in participation would indicate that 
administration of the demonstrations was not entirely successful. More importantly, each of these 
potential problems would suggest that the implemented demonstrations may be testing a service 
package that differs from the one intended, and that estimated demonstration net impacts should be 
interpreted cautiously. Second, knowing how many claimants participated in specific services and 
the types of claimants that participated is also important for interpreting estimated net impacts. If 
more or fewer claimants had participated in services or if different types of claimants had 
participated, then the estimated net impacts might have been different In addition, understanding 
what services claimants felt helped them the most is important for designing more efficient services 
for ongoing programs. 

We find that the demonstration was generally implemented as designed. A substantial proportion 
of claimants assigned to the structured JSA (SJSA) treatment attended each of the service 
components, and the timing of each component was generally as expected. A substantial proportion 
of claimants in the individualized JSA treatments (USA and USA+) attended the orientation and 
assessment interview, and the assessment interview occurred shortly after the orientation, as 
designed. Relatively few claimants in the individualized programs attended the job search workshop 
or testing. While it is not surprising that participation rates in the individualized programs were 
lower than rates in the structured program, since the services were provided on an as-needed basis 
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in the individualized programs, they were lower than expected. It was anticipated that demonstration 
staff would be more aggressive in assigning claimants to services. 

Relatively few claimants in the USA+ treatment were referred to or received training. 
Reluctance on the part of EDWAA staff to aggressively serve claimants in this treatment by 
providing greater access to limited training resources may explain this finding. The proportion of 
claimants who received training was nearly equal in all treatment groups, indicating that the 
EDWAA training component of the USA+ treatment was not fully implemented as designed. In both 
states, however, the proportion of claimants who received training was higher in the treatment 
groups than in the control group. Hence, the demonstration increased the frequency of EDWAA 
training receipt. Relatively few claimants in the demonstrations received services outside the 
demonstrations that were similar to those provided as part of the demonstrations. 

Overall, participation in services was higher in D.C. than in Florida. This difference in 
participation rates arose primarily because claimants in Florida went back to work more rapidly than 
claimants in D.C. Claimants in Florida were also less likely to be assigned to specific services and 
were more likely to be excused if they were assigned to a service than were claimants in D.C. 
However, the primary reason claimants did not attend services was because they were working, and 
the primary reason claimants were excused was because they were job attached (either found a job 
or had been placed on recall). Overall, failure to participate without being excused was very low. 

Demonstration participants were generally satisfied with the services they received. More than 
three-quarters of demonstration participants reported that they were very or somewhat satisfied with 
the JSA treatment they received. Participants in D.C. reported somewhat greater satisfaction than 
participants in Florida. There is no consistent difference in reported satisfaction by treatment group. 
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However, substantial proportions of claimants were not satisfied with the job search workshop for 
helping them find job openings. 

Participation in the JSA treatments and in specific services varied consistently along several 
claimant characteristics. Claimants with more education, job tenure, and prior employment in 
manufacturing industries and in professional occupations were more likely than other claimants to 
participate in the demonstration services in both Florida and D.C.—that is, claimants from these 
groups made up a larger proportion of the claimants participating in services than of the claimants 
not participating. They were also more likely to participate in each of the specific demonstration 
services. In contrast, claimants with prior employment in construction industries and blue-collar 
occupations were less likely to participate. 

We begin with a description of the services provided as part of the demonstrations and the 
process by which they were provided in each of the treatment groups. We then discuss assignment 
to and participation in services and the timing of services by treatment group for each state. 
Subsequent sections cover training services received outside of the demonstrations, claimants’ 
satisfaction with the services received, and finally, reasons claimants gave for not participating in 
assigned services. 



A. SERVICES PROVIDED IN JSA TREATMENTS 

The types of services demonstration participants received and the process by which participants 
were assigned to specific services varied across treatment groups. Claimants assigned to the SJSA 
package were expected to complete a preset sequence of services. After receiving a notification 
letter asking them to report to their local office for JSA services, SJSA participants began their 
treatment by attending a group orientation session at about the seventh week after filing their initial 
claim. After the orientation, participants in SJSA attended aptitude and interest testing. Following 




39 



68 



testing, claimants attended a job search workshop to help them formulate an adequate employment 
search plan. In an assessment interview following the workshop, a demonstration staff member 
reviewed the search plan and test results. Unless additional counseling was necessary, claimants in 
die structured treatment then proceeded with their self-guided job search. Periodic follow-up 
contacts with demonstration staff at four and eight weeks after orientation were also required. This 
sequence of activities was mandatory unless demonstration staff excused claimants from 
participation in the demonstration or from specific activities, or unless claimants returned to work. 

The USA and USA+ treatments used the same basic services available for SJSA. However, the 
method of service delivery was different, in that the individualized treatments were intended to target 
specific services to individuals based on their needs. At the orientation sessions, ISJA and ISJA+ 
claimants received an overview of the available services and signed up for an individual assessment 
interview, at which time an individual service .plan was developed. This plan was tailored to 
claimants and their specific situation. Demonstration services (such as testing, the job search 
workshop, or additional counseling and follow-up contacts) included in the individualized service 
plan were mandatory. 

Service delivery for USA+ participants followed the same individualized approach as USA, with 
die additional option of a training component to upgrade the skills of claimants who needed this 
service. The objective of this treatment was to create a one-stop-shopping environment where 
claimants would receive information about EDWAA training opportunities at the same office that 
provided job search assistance. An EDWAA staff member described EDWAA training opportunities 
during the orientation session. A Job Service staff member reviewed the available types of training 
with each claimant again during the assessment interview. Claimants interested in EDWAA training 
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were to meet with EDWAA staff to discuss the training opportunities that interested them. 
Participation in training services was strictly voluntary. 

B. ASSIGNMENT AND PARTICIPATION 

In this section, we present information on assignment to specific JSA services, excusal from 
attending services, and participation in services by state. Participation in the JSA treatments and in 
specific services varied consistently along several claimant characteristics. Among eligible 
claimants, those with more education, job tenure, and prior employment in manufacturing industries 
and in professional occupations were more likely to participate in the demonstration in both Florida 
and D.C. They were also more likely to participate in each of the specific demonstration services. 
In contrast, claimants with prior employment in construction industries and blue-collar occupations 
were less likely to participate. Older claimants were more likely to participate in Florida and less 
likely to participate in D.C., and blacks were more likely to participate in D.C. and less likely to 
participate in Florida. 

Because the process of assignment to services and the timing of services differ for the structured 
and individualized JSA groups, we discuss them separately. We begin with a presentation of results 
for the structured groups and then discuss results for the individualized groups. Data on participation 
in each service and the timing of services were collected on the PTS. 

1. Structured JSA 
a. Orientation 

Most claimants assigned to the JSA demonstration attended the initial orientation, but 
participation was substantially higher in D.C. than in Florida. As shown in the top panel of Table 
IV. 1, about 62 percent of the claimants assigned to the structured JSA program in Florida attended 
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TABLE III.l 



SERVICE ASSIGNMENT AND ATTENDANCE, STRUCTURED JOB SEARCH 
ASSISTANCE GROUPS BY STATE 





JSA Demonstration 
District of 

Columbia Florida 


New Jersey 
Demonstration 


Orientation 


Number assigned 


2,026 


3,032 




Percent excused 


4.8% 


28.2% 




Percent did not attend* 


16.4% 


9.7% 




Percent attended 


78.8% 


62.0% 


76.8% 


Testing 


Number assigned 


1,558 


1,653 




As percent of orientation attendees 


97.6% 


87.9% 




Percent excused of those assigned 


11.3% 


31.6% 




Percent did not attend of those assigned 


6.4% 


3.3% 




Percent attended of those assigned 


81.0% 


79.1% 




As percent of orientation attendees 


79.1% 


69.5% 


59.2% 


As percent of all orientation assignees 
Workshop 


62.3% 


43.1% 


45.5% 


Number assigned 


1,535 


1,743 




As percent of orientation attendees 


96.2% 


92.7% 




Percent excused of those assigned 


11.9% 


27.8% 




Percent did not attend of those assigned 


7.5% 


4.2% 




Percent attended of those assigned 


79.1% 


76.5% 




As percent of orientation attendees 


76.1% 


70.9% 


64.8% 


As percent of all orientation assignees 


59.9% 


43.9% 


49.8% 


Assessment 


Number assigned 


1,332 


1,471 




As percent of orientation attendees 


83.5% 


78.2% 




Percent excused of those assigned 


3.0% 


4.1% 




Percent did not attend of those assigned 


.8% 


.3% 




Percent attended of those assigned 


96.4% 


93.7% 




As percent of orientation attendees 


80.5% 


73.3% 


73.2% 


As percent of all orientation assignees 


63.4% 


45.5% 


56.2% 



Source: JSA demonstration participant tracking system; Corson, et al. (1989). 

Note: Percentages for those assigned may not add up to 100% because some claimants who did not attend the 

orientation attended services, because many claimants coded as excused were not coded as assigned to services, 
and because some claimants assigned had missing codes for attendance. 

■“Includes claimants who did not attend or who were still coded as rescheduled at the end of the demonstration. 
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the orientation, while nearly 80 percent of claimants assigned in D.C. attended the orientation. 

Attendance rates in Florida were lower than those observed for D.C. because claimants in 
Florida had shorter average spells of UI receipt than claimants in D.C. did, suggesting that a higher 
proportion of Florida claimants left UI prior to the orientation. Average duration on UI was about 
16 weeks in Florida versus 20 weeks in D.C. Most of those leaving UI returned to work. About 76 
percent of claimants in Florida who left UI prior to exhausting their benefits (excluding those 
excused from the demonstration) worked in the quarter following their exit. The corresponding 
figure for D.C. is about 60 percent. In addition, a larger proportion of the claimants in Florida was 
excused from the orientation than in D.C. More than 28 percent of the claimants assigned to the 
structured JSA program in Florida were excused, while less than 5 percent of those in D.C. were. 
The primary reason claimants were excused from the orientation session was because of re- 
employment or job recall. 

Compliance with the directive to attend the orientation was reasonably high in both states. Less 
than 10 percent of claimants assigned to the structured JSA program in Florida did not attend the 
orientation, while about 16 percent did not attend in D.C., as shown in Table III. 1 . After excluding 
those who were excused from the orientation, attendance rates in the two states are more similar. In 
Florida, 86 percent of those not excused attended orientation, as compared with 83 percent in D.C. 

b. Testing 

As expected, assignment to testing was almost universal in the structured JSA programs. As 

shown in the second panel of Table III.l, almost 98 percent of those who attended the orientation 

/"* 

in D.C. were assigned to testing, and about 88 percent of those who attended the orientation in 
Florida were assigned to testing. As was the case with orientation, a greater proportion of claimants 
in this treatment in Florida were excused from testing than in D.C. Almost one-third of the 
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claimants who were assigned to testing in Florida were excused, while only about 1 1 percent of those 
assigned to testing in D.C. were excused. 

Attendance, as a percentage of those assigned to testing, was fairly high in both states. About 
80 percent of those assigned to testing in both states attended. About 3 percent of those assigned to 
testing in Florida failed to attend, while about 6 percent in D.C. failed to attend. Because of a higher 
excusal rate, participation in testing among those who attended the orientation was lower in Florida 
than it was in D.C. About 70 percent of the claimants in Florida who attended the orientation 
received testing services. The comparable figure for D.C is 79 percent. 

c. Job Search Workshop 

Assignment to the job search workshop was nearly universal in the structured JSA program in 
both states. As shown in the third panel of Table IU.l, among those who attended the orientation, 
93 percent of claimants in Florida and 96 percent of claimants in D.C. were assigned to the job 
search workshop. As was the case with the orientation and testing, excusal from the workshop was 
more common in Florida than in D.C. More than one-quarter of claimants assigned to the workshop 
in Florida were excused from participating in the workshop, compared with about 12 percent in D.C. 
Very few claimants assigned to and not excused from the workshop failed to attend. In Florida, 4 
percent of those assigned failed to attend; in D.C. about 8 percent failed to attend. 

About three-quarters of the claimants in both states who were assigned to the workshop 
attended. Because of the higher excusal rate in Florida, the attendance rate at the workshop among 
those who attended the orientation was somewhat lower in Florida than in D.C. In Florida, 71 
percent of the claimants who attended the orientation attended the job search workshop, while in 
D.C. 76 percent attended the workshop. 
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d. Assessment 



Assignment to the assessment interview in the structured JSA program was somewhat lower 
than assignment to either testing or the job search workshop. As shown in the last panel of Table 
III.l, about 80 percent of claimants in both states who attended the orientation were assigned to an 
assessment interview, and about 95 percent of those assigned to assessment attended. Among those 
who attended the orientation, attendance at the assessment interview was somewhat lower in Florida 
than in D.C. Specifically, about 73 percent of the claimants who attended the orientation in Florida 
attended assessment, while over 80 percent attended assessment in D.C. 

e. Comparison with the New Jersey Demonstration 

The SJSA treatment was modeled after the New Jersey demonstration, which was conducted 
in 1986-1987. The SJSA treatment offered structured mandatory services similar to those offered 
in New Jersey. Given the similarities, we can make comparisons between the service participation 
rates in New Jersey, D.C., and Florida. This exercise can give us some idea of what variation might 
be expected among states in an ongoing program, as well as provide some context for interpreting 
any differences in net impacts between the demonstrations. 

The orientation attendance rate in D.C. (79 percent) was similar to that in the New Jersey 
demonstration (77 percent), while the Florida rate was lower (62 percent), as shown in Table III.l. 
As discussed earlier, the lower attendance rate in Florida can probably be attributed to the higher 
orientation excusal rate, which was largely due to more rapid re-employment. Attendance rates for 
services beyond orientation, as a percentage of all claimants assigned to the treatment, tended to be 
higher in the D.C. and lower in Florida than in New Jersey. For example, about 60 percent of D.C. 
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claimants participated in the job search workshop, compared with 50 percent in New Jersey and 44 
percent in Florida. 

2. Individualized JSA 

As discussed earlier, the ordering of services differed in the structured and individualized JSA 
treatments. In the individualized JSA treatments, the assessment interview occurred immediately 
after the orientation, before testing and the job search workshop, rather than after testing and the 
workshop, as was the case in the structured JSA treatments. Because of these differences, the order 
of presentation in this section differs from the section describing participation and timing of services 
in the structured JSA programs. 

a. Orientation 

Consistent with the findings for SJSA, attendance at the orientation in USA and USA+ was high 
for all groups and higher in D.C. than in Florida. As shown in the top panel of Table ni.2, more than 
three-quarters of those assigned to the orientation in D.C. attended the orientation, compared with 
66 percent in Florida. The comparable figures for the SJSA treatments were 79 percent for D.C. and 
62 percent for Florida. Thus, participation in the orientation was similar for all treatments in D.C., 
but somewhat higher for USA and USA+ than for SJSA in Florida. 

As was the case for SJSA, attendance in Florida was lower because claimants in Florida tended 
to have shorter UI spells, suggesting that a higher proportion of Florida claimants may have left UI 
before their scheduled orientatioa Moreover, a follow-up survey indicates that about 90 percent of 
those in Florida who did not attend the orientation reported that they had gotten a job before their 
scheduled orientation; about 78 percent of those surveyed in D.C. gave this reason. In addition, a 
substantially larger proportion of claimants in the individualized JSA treatments were excused from 
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TABLE III.2 



SERVICE ASSIGNMENT AND ATTENDANCE, INDIVIDUALIZED JOB SEARCH 
ASSISTANCE GROUPS, BY STATE 





District of Columbia 


Florida 






USA 


IJSA+ 


USA 


IJSA+ 


Orientation 

Number assigned 


2,002 


2,011 


3,007 


2,989 


Percent excused 


6.9% 


5.3% 


24.1% 


24.8% 


Percent did not attend* 


15.9% 


• 17.3% 


9.1% 


9.5% 


Percent attended 


77.2% 


77.4% 


66.7% 


65.8% 


Assessment 










Number assigned 


1,662 


1,587 


1,954 


1,913 


As percent of orientation attendees 


107.5% 


102.0% 


97.4% 


97.3% 


Percent excused 


6.1% 


3.8% 


3.9% 


3.6% 


Percent did not attend* 


.6% 


•3% 


.4% 


.4% 


Percent attended 


91.3% 


94.3% 


95.4% 


94.8% 


As percent of orientation attendees 


98.1% 


96.2% 


92.9% 


92.3% 


As percent of orientation assignees 


75.8% 


74.4% 


62.0% 


60.7% 


Testing 

Number assigned 


14 


18 


406 


314 


As percent of orientation attendees 


0.9% 


1.2% 


20.2% 


16.0% 


Percent excused 


0.0% 


11.1% 


13.8% 


14.6% 


Percent did not attend* 


7.1% 


0.0% 


3.0% 


4.5% 


Percent attended 


78.6% 


72.2% 


81.5% 


80.9% 


As percent of orientation attendees 


0.7% 


0.8% 


16.5% . . 


12.9% 


As percent of orientation assignees 


0.5% 


0.6% 


11.0% 


8.5% 


Workshop 

Number assigned 


28 


19 


328 


306 


As percent of orientation attendees 


1.8% 


1.2% 


16.3% 


15.6% 


Percent excused 


3.6% 


15.8% 


22.9% 


22.5% 


Percent did not attend* 


21.4% 


15.8% 


7.0% 


4.6% 


Percent attended 


60.7% 


42.1% 


74.4% 


79.4% 


As percent of orientation attendees 


1.1% 


0.5% 


12.2% 


12.4% 


As percent of orientation assignees 


0.8% 


0.4% 


8.1% 


8.1% 


Counseling 

Number assigned 


789 


767 


178 


142 


As percent of orientation attendees 


51.0% 


49.3% 


8.9% 


7.2% 


Percent attended 


95.6% . 


95.6% 


3.8% 


98.6% 


As percent of orientation attendees 


48.8% 


47.2% 


8.3% 


7.1% 


As percent of orientation assignees 


37.7% 


36.5% 


5.6% 


4.7% 



Source: JSA demonstration participant tracking system. 

Note: Percentages for those assigned may not add up to 100% because some claimants who did not attend 

the orientation attended services, because many claimants coded as excused were not coded as 
assigned to services, and because some claimants assigned had missing codes for attendance. 

* Includes claimants who did not attend or who were still coded as rescheduled at the end of the demonstration. 
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the orientation in Florida. Almost 25 percent of claimants in Florida were excused from the 
orientation, while only about 5 percent were excused in D.C. These figures are similar to those 
observed for claimants in the structured JSA programs. The primary reason claimants were excused 
was because they had obtained employment or been placed on job recall. 

Compliance with the directive to attend the orientation was high in both states. Less than 10 
percent of the claimants in the individualized JSA programs in Florida assigned to the orientation 
failed to attend, while about 15 percent failed to attend in D.C. After excluding claimants who were 
excused from attending the orientation, attendance rates in the two states are more similar. In 
Florida, almost 88 percent of those assigned to the orientation, who were not excused, attended. In 
D.C., about 82 percent of those who were not excused attended the orientation. These figures on 
compliance and attendance among those who were not excused are nearly identical to those observed 
for claimants in SJSA in each state. 

b. Assessment 

Assignment to the assessment interview was nearly universal in USA and IJSA+. As shown in 
the second panel of Table UI.2, in Florida, 97 percent of claimants who attended the orientation were 
assigned to assessment. In D.C., more claimants were assigned to assessment than attended the 
orientation, indicating that some claimants may have been assigned to an assessment interview 
without attending the orientation. These figures are higher than those observed for claimants in 
SJSA, as was expected given that the assessment interview occurred much earlier in USA and USA+ 
than it did in SJSA. 

Consistent with the findings for SJSA, very few claimants assigned to assessment in USA or 
USA+ were excused or failed to attend the assessment interview. In both D.C. and Florida, the 
attendance rate exceeded 90 percent. Attendance among those not excused exceeded 95 percent. 
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These figures are nearly identical to those observed for SJSA and show that there was nearly 
universal compliance with the directive to attend the assigned assessment interview. One reason 
attendance at assessment was high among claimants who attended orientation was that assessment 
sometimes happened on the same day as orientation in the IJSA and USA+ treatments. 

Attendance, as a percentage of those attending the orientation, is somewhat higher in D.C. than 
Florida. About 97 percent of claimants who attended the orientation attended assessment, while the 
comparable figure for Florida is about 92 percent. Attendance rates at the assessment interview are 
lower in SJSA (73 percent for Florida and 81 percent for D.C.) than in USA and IJSA+ in both D.C. 
and Florida. This finding is not surprising. As noted at the beginning of the chapter, assignment to 
the assessment interview occurred later in the structured JSA treatment than it did in the 
individualized JSA treatments. For this reason, more claimants in SJSA were expected to exit UI 
before the assessment interview. 



c. Testing 

Assignment to testing was veiy low in the individualized JSA programs. As shown in the third 
panel of Table m.2, about 20 percent of claimants in USA who attended the orientation were 
assigned to testing in Florida, and about 16 percent in IJSA+ were assigned to testing. About 1 
percent of claimants in the individualized JSA programs in D.C. who attended the orientation were 
assigned to testing. These figures for testing assignment are much lower than those for SJSA. 

It is not surprising that assignment to testing is much lower in the individualized treatments. 
Testing was part of the package of services provided to all claimants in SJSA, while in the 
individualized treatments, only claimants who were determined to need testing were assigned to this 
service. Assignment to testing in the individualized treatments was lower in D.C. than in Florida. 
Staff in D.C. did, however, provide one-on-one counseling to many claimants (see the bottom panel 
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of Table ni.2). D.C. staff may not have aggressively assigned claimants to group services because 
they felt that one-on-one counseling was more effective or more acceptable to demonstration 
participants and because of a shortage of resources for providing group services. The D.C. office 
had difficulty maintaining sufficient staff who were trained to conduct group services and had a 
shortage of space for providing group services. In contrast, the D.C. office had ample staff for one- 
on-one counseling and adequate office space for conducting one-on-one services. 

As was the case with assessment, very few claimants assigned to testing were excused or failed 
to attend. Consequently, attendance among those assigned to testing was high. About 80 percent 
of those assigned to testing in the individualized treatments attended. The only exception is USA+ 
in D.C., where 72 percent of those assigned to testing attended. • However, figures for the D.C. 
programs are based on small numbers and should be interpreted with caution. These figures show 
that there was high compliance with the directive to attend testing. Among those not excused, 
attendance is very high. Almost 95 percent of those not excused in Florida attended testing, and 
about 80 percent of those not excused in D.C. attended testing. Attendance rates, as a percentage 
of claimants attending the orientation, indicate that testing was rare in the individualized JSA 
programs. Among those attending the orientation, about 15 percent of claimants in Florida received 

V' 

testing services, and less than 1 percent were tested in D.C. 

d. Job Search Workshop 

As was the case with testing, assignment to the job search workshop was relatively rare in the 
individualized JSA treatments. As shown in the fourth panel of Table HI.2, about 16 percent of 
claimants in the individualized treatments in Florida and less than 2 percent of claimants in D.C. 
were assigned to the workshop. Considerably fewer claimants were assigned to the workshop in 
IJSA and USA+ than were assigned in SJSA, because only claimants deemed in need of the job 
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search workshop were assigned in the individualized programs. Again, assignment to the workshop 
was lower in the individualized treatments in D.C than in Florida. However, D.C. conducted one-on- 
one counseling for many claimants in the individualized treatments and may have provided claimants 
with information similar to that provided in the workshop. 

Almost a quarter of those assigned to the job search workshop in the Florida individualized 
treatments were excused from attending. About three-quarters of those assigned to the workshop in 
Florida attended, and almost 100 percent of those not excused attended the workshop in Florida. 
These figures are similar to those observed for claimants in SJSA in Florida. 

Attendance rates, as a percentage of those attending the orientation, are low for the 
individualized JSA treatments. About 12 percent of those who attended the orientation attended a 
workshop in Florida, and about 1 percent attended in D.C. These results are similar to those reported 
above for testing in USA and USA+ and are much lower than those found for SJSA. 

e. Counseling 

In addition to the services discussed above, claimants in the individualized JSA programs, who 
were deemed to be in need, could be assigned to additional one-on-one counseling. Assignment to 
counseling was much more common in D.C. than in Florida. As shown in the last panel in Table 
III.2, less than 10 percent of claimants who attended the orientation in Florida were assigned to 
additional counseling, compared to about 50 percent of those in D.C. As noted earlier, in D.C. 
claimants were more likely to receive one-on-one services instead of group services like testing and 
the job search workshop. This difference in how claimants were- assigned to services is important 
for interpreting state differences in estimated net impacts for the individualized JSA programs. 

As was the case with other services, attendance among those assigned to counseling was very 
high, 94 percent or higher. Attendance rates, as a percentage of claimants attending the orientation. 



were much higher in D.C. Almost one-half of the these claimants in the individualized programs 
in D.C attended additional counseling, compared with less than 10 percent in Florida. So D.C. 
appears to have compensated for the lack of group services among the USA and IJSA+ claimants 
by providing a large volume of one-on-one counseling. 

C. TIMING OF SERVICES 

The demonstrations were successful in achieving early intervention. The timing of initial 
services in the JSA treatments was generally consistent in both states with the original demonstration 
design. This was true for both the structured and individualized treatments. 

1. Structured JSA 

As shown in Table III.3, the mean time between the beginning of the benefit year and the 
orientation was 49 days (7 weeks) in Florida and 46 days (6.6 weeks) in D.C., both very close to the 
expected time of seven weeks. About 80 percent of claimants in each state attended the orientation 
within 8 weeks or less, and very few claimants had lengthy delays before the start of participation. 
Only 6 percent in Florida and 3 percent in D.C. attended the orientation more than 10 weeks after 
the beginning of their benefit year. 

The longer time to services in Florida arose partly because of design differences between the 
two demonstrations. Not all Florida demonstration offices had enough participants to offer initial 
demonstration services on a weekly basis. For this reason, services were delivered on a bi-weekly 
schedule in Florida. A bi-weekly schedule implies that half of claimants in Florida waited an extra 
week before participating in the orientation. In D.C., demonstration services were provided on a 
weekly basis. This difference in the scheduling of services accounts for the three-day difference in 
the timing of the orientation. 
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TABLE IH.3 



TIMING OF SERVICES, STRUCTURED JOB SEARCH ASSISTANCE 

GROUPS, BY STATE 



Time Between Services 


JSA Demonstration 
District of 

Columbia Florida 


New Jersey 
Demonstration 


Beginning of Benefit Year to Orientation 


Less than 6 weeks 


42.9% 


22.1% 




6 to 8 weeks 


42.1% 


55.9% 




8 to 10 weeks 


12.2% 


16.5% 




More than 10 weeks 


2.8% 


5.5% 




Mean in days 


46 


49 


35 


Orientation to Testing 


Less than 1 week 


15.8% 


87.2% 




1 to 2 weeks 


75.1% 


4.2% 




More than 2 weeks 


9.1% 


8.6% 




Mean in days 


9 


3 


3 


Testing to Workshop 


Less than 1 week 


92.9% 


89.2% 




1 to 2 weeks 


4.4% 


1.2% 




More than 2 weeks 


2.7% 


9.6% 




Mean in days 


2 


5 


8 


Workshop to Assessment 


Less than 1 week 


76.9% 


33.1% 




1 to 2 weeks 


16.5% 


57.2% 




More than 2 weeks 


6.6% 


9.7% 




Mean in days 


5 


8 


4 



Source: JSA demonstration participant tracking system; Corson et al. (1989). 
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Following orientation, the remaining services occurred in quick succession. More than 90 
percent of claimants attended testing within two weeks of their orientation. After testing, about 90 
percent attended the job search workshop within less than one week. Assessment then occurred 
within about a week after the workshop. 

Services were generally initiated earlier in a claimant’s UI spell in the New Jersey demonstration 
than in the JSA demonstration. In the New Jersey demonstration, the orientation was targeted for 
the fifth week of a claimant’s UI spell, while in the JSA demonstration the orientation was designed 
to occur in about the seventh week. Average time from the beginning of the benefit year to the 
orientation was only 35 days in the New Jersey demonstration, which is more than one week shorter 
than the corresponding period in D.C. or Florida. There are two reasons why services began earlier 
in New Jersey. First, UI staff in Florida and D.C. felt they needed two weeks to mail the letter telling 
claimants to attend the orientation, while New Jersey did it in one week. Second, a bi-weekly 
schedule for providing services was used in Florida, while New Jersey and D.C. used a weekly 
schedule. The remaining mandatory services were delivered shortly after orientation in all three 
states. 

2. Individualized JSA 

The timing of initial services in the individualized JSA treatments was consistent with the 
original demonstration design. As shown in Table U1.4, the mean time from the beginning of the 
benefit year to the USA orientations was identical to that for SJSA--49 days in Florida and 46 days 
in D.C. As was the case in the structured programs, about 80 percent attended the orientation within 
8 weeks of the beginning of the benefit year and very few had lengthy delays: only about 5 percent 
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TABLE III.4 



TIMING OF SERVICES, INDIVIDUALIZED JOB SEARCH 
ASSISTANCE GROUPS, BY STATE 





District of Columbia 




Florida 


Time Between Services 


IJSA 


IJSA+ 


IJSA 


USA+ 


Beginning of Benefit Year to Orientation 
Less than 6 weeks 


36.0% 


41.4% 


21.7% 


22.5% 


6 to 8 weeks 


50.1% 


42.4% 


56.7% 


56.3% 


8 to 10 weeks 


12.2% 


11.8% 


16.4% 


16.5% 


More than 10 weeks 


1.7% 


4.4% 


5.2% 


4.7% 


Mean in days 


46 


46 


49 


49 


Orientation to Assessihent 
Less than 1 week 


90.1% 


85.4% 


84.7% 


88.9% 


1 to 2 weeks 


7.6% 


8.5% 


1 1.7% 


8.7% 


More than 2 weeks 


2.3% 


6.1% 


3.6% 


2.4% 


Mean in days 


2 


4 


3 


3 



Source: JSA demonstration participant tracking system. 
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in Florida and about 3 percent in D.C. had delays of more than 10 weeks. The longer delay found 
for Florida is partly due to the bi-weekly service delivery used in that state. The time between the 
orientation and assessment was short -- an average of three days in both Florida and D.C. (two days 
for Treatment 2 and four days for Treatment 3). Very few claimants waited more than two weeks. 

D. TRAINING 

In addition to the other services, claimants could also receive on-the-job and classroom training. 
Participation in training services was strictly voluntary. In contrast to other groups, claimants 
assigned to the USA+ group received more information about training programs, received their 
information directly from EDWAA staff, and were intended to have greater access to training service 
resources. However, as shown in Table IH.5, relatively few claimants in the individualized program 
with training were referred to training. In Florida, about 8 percent of those who attended the 
orientation were referred to training. About 14 percent were referred to training in D.C: 

Attendance among those referred to training is relatively low compared to participation in other 
services, especially for D.C. About 64 percent of those referred to EDWAA in Florida received 
training, while only about 12 percent of those referred in D.C. received training. The low rates are 
not too surprising because there are many reasons that claimants would have dropped out between 
referral and potential participation in EDWAA. For example, referred claimants might have been 
found ineligible for EDWAA training, failed to find an available training slot or a training 
opportunity that was convenient for them, or decided that training was not appropriate for them. The 
especially low rate in D.C. may indicate that there were considerable obstacles to EDWAA training 
in that state. 

Overall, although EDWAA training participation among the demonstration participants was low, 
the demonstration increased the training rates. In Florida, 2.8 percent of the control group 
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TRAINING ASSIGNMENT AND ATTENDANCE BY JOB SEARCH ASSISTANCE GROUP AND STATE 
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participated in training compared with 3.3 to 3.5 percent of the treatment groups. The difference in 
the training rate for the control group and the combined treatment groups is statistically significant 
at the 90 percent confidence level. The training rates in D.C. are lower than in Florida, but again the 
rates are higher for the treatment groups than for the control group. Only 0.8 percent of the control 
group in D.C. participated in training, compared with 1.2 to 1.4 percent for the treatment groups. 
Again, these differences are statistically significant at the 90 percent level. 

Although the treatments had a positive effect on training participation, the training rate was not 
higher in the IJSA+ group than in the other treatment groups, despite the USA+ design that 
emphasized access to EDWAA training. This finding confirms the perceptions from the site visits 
that EDWAA staff were not providing claimants in the IJSA+ greater access to training resources. 
In effect, the DSA+ treatment was not substantially different than the USA treatment in the field. 

Results from the follow-up survey of claimants in all three treatments groups and in the control 
groups underscore the demand for more training. 1 About 60 percent of the claimants surveyed in 
Florida and about 70 percent of those surveyed in D.C. indicated they were interested in on-the-job 
training. Only 9 percent of those in Florida and 15 percent of those in D.C. interested in on-the-job 
training reported they had been offered training. About 50 percent of those in Florida offered on-the- 
job training attended training, while about 30 percent of those offered in D.C. attended. Overall, 
only 5 percent of those in Florida and D.C. who said they were interested in on-the-job training 
actually received such training. 



'The follow-up survey was conducted by phone approximately one year after a claimant’s initial 
UI claim. It included questions about service participation, job search activities, pre-UI employment 
and earnings, post-UI employment and earnings, and personal characteristics. Appendix A discusses 
the results of the survey. 
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Nearly identical figures are reported for classroom training. About 60 percent in Florida and about 
70 percent in D.C. said they were interested in classroom training. About 20 percent of those interested 
in Florida and about 33 percent of those interested in D.C. said they were offered classroom training. 
About 50 percent of those offered classroom training in Florida attended. The comparable figure for 

D. C. is 44 percent. Overall, about 10 percent of those interested in classroom training in Florida and 
about 15 percent of those interested in D.C. actually received classroom training. 

* 

E. SERVICES RECEIVED OUTSIDE THE DEMONSTRATION 

Data on re-employment services received outside the demonstration were collected as part of the 
follow-up survey, which is described in Appendix A. These data are used to determine whether 
claimants in the control groups received more outside services than claimants in the treatment groups. 
The level of services each group received is important for interpreting the estimated net impact. If the 
control group received greater outside services there could be smaller estimated net impacts. 

Information on the receipt of services outside the demonstration is displayed in Table III.6. Except 
for training and education, the results in Table in.6 show that claimants in the control group received 
more outside services than claimants in the treatment groups. For instance, about 13 percent of 
claimants in the control group in Florida report receiving outside services, while 3 to 5 percent of 
claimants in the treatment groups report outside services. In D.C., 17 percent of claimants in the control 
group and 4 to 6 percent of claimants in the treatment groups report receiving outside services. 
Similarly, 6 to 7 percent of claimants in the control group in both states report receiving JTPA services, 
while 1 to 2 percent of the claimants in the treatment groups in both states report receiving JTPA 
services. This same pattern is found in reports of receipt of specific services. 

As is clear from these statistics, the control group reported receiving substantial services outside the 
demonstration. The comparison of outcomes for the treatment and control groups, therefore, 
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OUTSIDE SERVICES BY JOB SEARCH ASSISTANCE GROUP AND STATE 
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provides an estimate of the impacts of the three JSA service packages compared with outside services 
rather than compared with no services. If all services tended to reduce UI spells, greater receipt of 
outside services by control group members suggests that the estimated net impacts of the demonstration 
were smaller than they would have been had claimants in the control groups not received greater outside 
services. However, the proportion of control group members receiving outside services is still low 
relative to the proportion of treatment group members who received services either inside or outside the 
demonstration. Consequently, the elfect of the availability of outside services on the estimated net 
impacts of the demonstration should have been small. 

The second to last row of Table DI.6 shows that claimants in the control groups were no more likely 
to receive training services outside the demonstration than were claimants in the treatment groups. The 
last row shows that controls did not participate in appreciably more training programs outside of the 
demonstration than did treatments. These findings suggest that receipt of outside training will have little 
effect on the estimated net impacts. 

F. CUSTOMER SATISFACTION 

The follow-up survey contained several questions designed to assess participants’ satisfaction with 
the services they received as part of the JSA demonstration. 2 As shown in Table III. 7, about one-quarter 
of those in Florida and about a third of those in D.C. who attended an assessment interview thought that 
it was very useful. About 40 percent in Florida, and about one-third in D.C., said the assessment 
interview was somewhat useful. About a third in each state said that the interview was not useful. 
Although the assessment interview was somewhat different in the structured and individualized JSA 
treatments and more claimants in the individualized treatments 
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2 The follow-up survey is described in Appendix A. 
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TABLE III. 7 



CUSTOMER SATISFACTION BY JOB SEARCH ASSISTANCE GROUP AND STATE 





District of Columbia 




Florida 






SJSA 


IJSA 


IJSA+ 


SJSA 


IJSA 


IJSA+ 


Usefulness of assessment for planning 
work future 


Percent very 


34.1 


27.3 


32.3 


25.9 


25.2 


28.6 


Percent somewhat 


32.2 


34.1 


35.3 


42.6 


35.2 


37.8 


Percent not at all 


33.6 


38.6 


32.3 


31.6 


39.6 


33.6 


Usefulness of workshop for finding job 
opening 


Percent very 


26.7 


23.6 


15.3 


22.2 


17.3 


21.8 


Percent somewhat 


33.1 


34.7 


35.6 


34.7 


39.1 


26.8 


Percent not at all 


40.3 


41.6 


49.2 


43.1 


43.7 


51.5 


Usefulness of workshop for resume 


Percent very 


46.4 


24.6 


17.3 


37.6 


29.4 


31.4 


Percent somewhat 


27.9 


26.1 


24.1 


30.1 


24.8 


29.4 


Percent not at all 


25.7 


49.4 


58.6 


32.3 


45.8 


39.2 


Usefulness of workshop in helping define 
job skills and goals 


Percent very 


38.3 


18.9 


25.4 


34.6 


29.1 


27.5 


Percent somewhat 


32.0 


43.2 


25.4 


32.2 


29.1 


36.3 


Percent not at all 


29.7 


37.8 


49.2 


33.2 


41.8 


36.3 


Usefulness of Job Search Assistance for 
finding a job 


Percent very 


32.2 


31.2 


33.9 


30.4 


25.7 


29.7 


Percent somewhat 


50.1 


47.6 


45.3 


43.5 


52.6 


47.5 


Percent not very 


11.1 


18.0 


9.5 


19.2 


12.9 


16.8 


Percent not at all 


6.6 


3.2 


11.3 


7.0 


8.9 


5.9 



Source: JSA demonstration follow-up survey. 
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attended assessment than in the structured programs, there are no substantial treatment group differences 
in satisfaction with the assessment interview. 

This same general pattern is observed for satisfaction with the job search workshop in helping 
participants find jobs. Although respondents report finding the workshop somewhat less helpful for 
finding a job than the assessment interview, there are no substantial differences in reports of the 
usefulness of the workshop for helping claimants find a job among treatment groups. In contrast, 
claimants in the structured JSA programs report greater satisfaction with the workshop with respect to 
helping them write a resume and helping them define their job skills and goals. The treatment group 
differences are somewhat larger in D.C. than in Florida. For instance, more than 45 percent of 
respondents in the structured program in D.C. report the workshop was very useful for help with resume 
writing, while about 20 percent of respondents in the individualized programs reported that the workshop 
was very useful for helping them with resume writing. This finding is somewhat surprising because 
many more claimants in the structured programs were required to attend the job search workshop. 

As shown at the bottom of Table III.7, about 30 percent of claimants in both Florida and D.C. 
thought that JSA was very useful in helping them find a job. Almost 50 percent in both Florida and D.C. 
said that JSA was somewhat useful in helping them find a job. About 20 percent said that JSA was not 
very useful or not at all useful in helping them find a job. There are no substantial differences in 
reported satisfaction by treatment group. This finding is somewhat surprising because claimants in the 
structured programs were assigned to and received more services than claimants in the individualized 
programs. 



G. REASONS FOR NOT ATTENDING 
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The follow-up survey also contained questions designed to examine reasons for service non- 
attendance. As shown in Table HI.8, the primary reason that claimants gave for not attending the 
orientation was they had found a job before orientation. In Florida, 80 to 90 percent of nonattendees 
reported they found a job before their scheduled orientation, while in D.C. about 75 percent gave this 
response. The figures are somewhat lower for the Structured program in Florida, and for the 
individualized program with training in D.C. About 5 percent in each state reported that they did not 
attend the orientation because they didn’t think JSA would help them get a job. 

Employment was also the most common reason claimants gave for not attending their scheduled 
job search workshop. This reason for not attending the workshop was given most often by claimants 
in the structured programs and given least often by claimants in the individualized program that included 
training. Because attendance rates for the workshop were high, relatively few respondents answered this 
question. Consequently, any treatment group differences must be interpreted with caution. About 20 
percent in both Florida and D.C. report that they didn’t attend the workshop because they didn’t think 
it would help them find a job. 

Employment was also the most common reason given for not attending the assessment interview. 
Almost 55 percent in Florida and 45 percent in D.C. report that they didn’t attend assessment because 
they had gotten a job before the date of their assigned assessment interview. About a third of the 
claimants in Florida and D.C. report that the reason they did not attend was that they thought that the 
assessment interview wouldn’t help them get a job. Because attendance rates for assessment were high, 
relatively few respondents were asked these questions, and for this reason, any treatment group 
differences should be interpreted with caution. 
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TABLE III.8 



REASONS FOR NOT ATTENDING SERVICE BY JOB SEARCH ASSISTANCE 

GROUP AND STATE 





District of Columbia 




Florida 






SJSA 


IJSA 


IJSAf 


SJSA 


IJSA 


IJSA+ 


Orientation 


Percent had job 


79.6 


78.3 


71.4 


81.5 


88.7 


90.8 


Percent on recall 


16.3 


8.7 


21.4 


4.2 


1.9 


4.6 


Percent in other program 


2.0 


4.4 


o.o - 


7.6 


1.9 


1.5 


Percent said service wouldn’t help 


2.0 


8.7 


7.1 


6.7 


7.6 


3.1 


Workshop 


Percent had job 


46.2 


25.9 


12.3 


34.9 


23.4 


18.4 


Percent on recall 


2.5 


3.7 


14.0 


4.8 


3.1 


3.3 


Percent in other program 


2.5 


14.8 


5.2 


19.1 


11.1 


6.7 


Percent other excused 


18.0 


7.5 


5.2 


15.9 


1.6 


8.4 


Percent said service wouldn’t help 


15.4 


24.1 


24.6 


7.9 


31.3 


26.7 


Percent other 


15.4 


24.1 


38.6 


17.5 


29.3 


36.7 



Assessment 
Percent had job 


51.9 


30.0 


28.6 


65.0 


39.1 


50.0 


Percent on recall 


11.1 


20.0 


28.6 


5.0 


8.7 


0.0 


Percent in other program 


7.4 


10.0 


14.3 


5.0 


8.7 


7.1 


Percent said service wouldn’t help 


25.9 


40.0 


28.6 


25.0 


43.5 


42.9 



Source: JSA demonstration follow-up survey. 
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IV. COSTS OF THE INTERVENTION 



An important aspect of the demonstration evaluation is to document and assess the costs of the three 
treatments. This information is essential for determining the degree to which each of the treatments would 
be cost-effective on an ongoing basis. Moreover, cost information is essential for planning and budgeting 
future programs. 

Our primary objective in this chapter is to measure the costs of each treatment and to compare costs 
across the treatments and states. We attempt to explain costs differences between the states by linking 
these cost differences to differences in services in the two states. Our comparisons will also reveal 
whether, as expected, the individualized JSA treatments (USA and IJSA+) were less costly than the 
structured JSA treatment (SJSA) and the magnitude of the cost difference. The findings on costs presented 
in this chapter are also used as a basis for our evaluation of the cost-effectiveness of the JSA treatments, 
which is presented in Chapter X. 

A. ESTIMATING PROGRAM COSTS 

The cost of operating the demonstration included the local office cost of providing the initial services 
via the Job Service and the cost to EDWAA of having an EDWAA staff member involved in the 
demonstration presentations and assessments. There are also costs from having UI staff monitor 
compliance of claimants with the reporting requirements. Costs were also incurred in the central offices 
of D.C. and Florida to manage the demonstration, monitor the sites, and provide training to staff. 

For both states, we have cost data that are based on staff time and other expenses that were charged 
to a demonstration account. These charges cover staff training, the pilot phase of the demonstration, and 
die full operation of the demonstration. In Florida both central office costs and local office costs were 
charged to the demonstration, so the total charges represent the full costs of the demonstration. In contrast, 
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D.C. did not charge central office costs directly to the demonstration— only demonstration office staff 
charged to the demonstration. In the remainder of our cost analysis, we assume that D.C. central office 
costs are equal to 18 percent of local office costs, which is the same assumption D.C. used in its accounting 
of program costs. EDWAA staff in either site did not charge their time to the demonstration, so we 
generate separate estimates of EDWAA costs. 

Total demonstration charges in the two states were equal to $1,301,267 in D.C. and $1,356,659 in 
Florida. Our objective in this chapter is to transform these charges into measures of demonstration costs 
per claimant for each of the treatments in the two states. Ideally, we would like to generate estimates of 
costs per claimant of providing similar services to the treatments in an ongoing program. However, we 
have limited information for using our cost data to generate -accurate estimates of the costs of an ongoing 
program. We have chosen, therefore, to calculate the costs of the intervention as it occurred in the 
demonstration, and we treat these as our baseline estimates of providing the service packages in an ongoing 
program. In our sensitivity analysis, we estimate the effect of various changes in our assumptions on the 
baseline estimates. These changes in assumptions are designed to account for ways in which costs of an 
ongoing program might differ from costs of the demonstration. 

The first step in transforming the demonstration charges into per-claimant cost figures is to determine 
how the pilot phase should be treated in the transformation. In Florida, the pilot phase operated in a single 
office (Orlando). Accounting for the costs of the pilot phase in Florida is straightforward because the pilot 
phase operated in the same manner and with the same intensity as the full demonstration operations 
throughout the state. But accounting for the costs of the pilot phase in D.C. is less straightforward. In D.C., 
die pilot phase used the same office and staff as the full demonstration operations, but the pilot phase 
served only about a quarter of the claimants per week that were served during the full demonstration 
operations. Given this difference, the costs per claimant served during the pilot phase may have been 
different than the costs per claimant during the full demonstration operations. 
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Because of the different intensity of the pilot phase in D.C., we estimate costs for D.C. using two 
different approaches. Our primary estimates assume that the per-claimant costs were the same for the pilot 
phase as for the full demonstration. We also present alternative estimates that are based on the assumption 
that total costs per week of serving the pilot claimants were the same as the total costs per week of serving 
the full demonstration claimants, regardless of how many claimants were served in either case. In this 
case, the per-claimant costs would be higher during the pilot phase than during full demonstration 
operations because during the pilot phase fewer claimants were served for the same weekly costs. We will 
present the estimated costs per claimant for the full demonstration operations as our alternative estimates. 

The second step in calculating per-claimant costs is to allocate the costs to the different treatments. 
Although each state charged costs directly to the demonstration, costs were not charged to separate 
treatments. Allocation of costs to different treatments is a key issue, however, because the individualized 
treatments (IJSA and USA+) were expected to be less expensive than the structured JS A treatment (SJSA). 
If some claimants did not need all of the services offered in SJSA, then an individualized approach, which 
tailors the services to claimants’ needs, should be a cheaper method of providing services. To examine 
the evidence on this point, we would like to be able to estimate costs for the separate treatments to 
document the cost advantage of the individualized treatments. Since our data provide only cost figures for 
the demonstration as a whole, we need to develop a method for allocating the costs to treatments. Our 
chosen method is to estimate the staff time that was spent serving the average claimant in each of the 
treatment groups. Then based on the relative time spent serving claimants in the different treatment 
groups, we allocate the reported costs to the different treatments. 

The estimates and assumptions used to allocate the demonstration costs to treatments were adapted 
to reflect the different approaches used to operate the demonstration in the two states. These figures, which 
are presented in Table IV. 1, are the basis for our calculations of the staff time per claimant required to 
provide each service within each treatment. For example, Table IV. 1 shows that the SJSA orientation in 
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D.C. was conducted by two JSA staff members, and we estimate that the average orientation lasted 0.75 
hours. We assume that staff spent time on paperwork outside orientation that was approximately equal 
to time spent inside orientation. Performing the orientation therefore required 1.5 hours of two staff 
members’ time, which yields total staff time needed to conduct the orientation of 3 person-hours. Since 
average attendance at a D.C. SJSA orientation was about 30 claimants, the orientation costs per attendant 
were 0.1 hours (3 person-hours divided by 30 claimants): For claimants who failed to attend the 

orientation, we assume time spent following up on them was equal to 0.25 hours per claimant. Given that 
the orientation attendance rate was 76 percent, the average claimant had a .76 chance of attending and 
having costs equal to 0.1 hours. Alternatively, the average claimant had a .24 chance of not attending and 
having costs equal to 0.25 hours. The orientation cost for the average claimant was therefore 0.136 person- 
hours [(.76 * 0.1)+(.24 * 0.25)]. 
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ESTIMATES AND ASSUMPTIONS USED TO ALLOCATE 
DEMONSTRATION COSTS TO TREATMENTS 
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is, we assume a one hour orientation requires one hour of paperwork. 



We made similar calculations for all of the services for each treatment. The estimates for each service 



were summed to yield an estimated total staff time spent per claimant for each treatment. These treatment 
totals were then used to allocate the actual charges across the treatments. 

Some of the differences between D.C. and Florida that are shown in Table IV. 1 are the result of 
different operations in the two states. For example, in Florida, claimants were offered services in their 
local offices, so the number of claimants participating in group services was relatively small in the offices 
with small caseloads. In D.C., all claimants were offered services in a centralized demonstration office, 
and a large number of claimants were served each week. Hence, the number of participants in the typical 
group service, such as testing or the job search workshop, tended to be higher in D.C. than in Florida. For 
example, the estimated average attendance at testing was 10 claimants in D.C. compared with 5.5 claimants 
in Florida. Similarly, the estimated average attendance at the job search workshops was 25 claimants in 
D.C. compared with 5.5 claimants in Florida. 

The difference in attendance rates between the states is especially important in estimating the staff 
time required for group services. Given this difference, the costs of the group services in SJSA were 
spread across a larger number of claimants in D.C. than in Florida, and this has some important 
implications for differences in the estimated costs per claimant in D.C. and Florida. First, the staff time 
per claimant to provide group services tended to be lower in D.C. than in Florida. Hence, the costs of 
SJSA, which was composed largely of group services, tended to be lower relative to USA and USA+ in 
D.C. than in Florida. 

B. COST ESTIMATES 

Using the estimates and assumptions presented in Table IV. 1, we estimated staff time per claimant 
for each of the services in each treatment. By combining these estimates of the different services, we 
derived an estimate of staff time used per claimant for each treatment group. The total demonstration costs 
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were then allocated to the treatments in proportion to the relative estimates of staff time needed for each 
treatment. These per-claimant cost estimates are presented in Table IV.2. 

As expected, the costs per claimant of SJSA exceeded the costs of USA and IJSA+ in both states. In 
D.C., the SJSA costs were S286 per claimant, compared with $199 and $216 for IJSA and IJSA+. In 
Florida, the SJSA costs were $241 per claimant, compared with $97 and $103 for USA and IJSA+. 

These estimates also imply that the D.C. demonstration was more costly than the Florida 
demonstration. All of the D.C. treatments were more expensive than the Florida counterparts. The primary 
reason for the difference probably is that staff are more highly paid in D.C. than in Florida, which at least 
partly reflects the higher cost of living in D.C. The D.C. treatments also tended to have higher 
participation rates (see Chapter III), so more services were provided in D.C. Furthermore, D.C. reported 
providing substantial one-on-one counseling, which may have further added to the resources used in the 
demonstration. This high level of services translated into large amounts of staff time charged to the 
demonstration. D.C. also spent significant staff time following up on claimants assigned to USA or USA+ 
after they had completed their scheduled services. In contrast, Florida conducted no follow-up with USA 
or USA+ claimants. 1 ' 

The differences in costs between SJSA and the two USA treatments were larger in Florida than D.C. 
In dollar terms, SJSA costs were about $140 higher than the average of the USA and USA+ costs in 
Florida, while the comparable difference in D.C. was roughly $80. In percentage terms, SJSA costs were 
about 140 percent higher than the USA and USA+ costs in Florida, compared with 



1 Florida chose not to follow up with USA and USA+ claimants because no follow-up of these 
claimants was specified in the original demonstration design. D.C. independently chose to follow 
up with these claimants as a way to extend the demonstration design. 
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about 37 percent higher in D.C. This was expected because the Florida group services included fewer 
participants, so the costs of SJSA, which emphasized group services, were spread over a smaller group in 
Florida than in D.C. This outcome translates directly into higher relative per- claimant costs in Florida 
than in D.C. 

The largest cost component of any treatment was associated with local office staff. In D.C., the cost 
for local office staff to provide SJSA was $209 per claimant, which was 73 percent of the total SJSA costs. 
Similarly, in Florida, the cost for local office staff to provide structured JSA was $136, 56 percent of the 
total SJSA costs. Nonlabor costs associated with local operations ranged from $20 per claimant for IJSA 
to $69 for SJSA in Florida. In D.C. the range was from $27 for IJSA to $42 for SJSA. 

Costs associated with the central office in each state were a substantial proportion of total costs. In 
allocating the costs to treatments, we assumed that central office costs per claimant were equal among the 
three treatments. In D.C., the estimated central office costs were $35 per claimant, which was derived 
using the generic rate (18 percent) that D.C. applies to direct costs to estimate central office costs in its own 
accounting system. 2 In contrast, the estimates of Florida central office costs were based on central office 
staff time and other expenditures that were charged directly to the demonstration. Estimated central office 
costs in Florida were $36 per claimant, which is nearly identical to D.C. For SJSA, the central office costs 
represent 12 percent of total costs in D.C. and 15 percent of total costs in Florida. The central office costs 
represent a larger proportion of the individualized treatments- 18 and 16 percent for USA and USA+ in 
D.C. and 37 and 35 percent for the USA and USA+ in Florida. 

The data presented in Table IV.2 also allow us to compare the costs of SJSA in D.C. and Florida with 
the costs of the similar structured JSA treatment in the New Jersey Unemployment Insurance Re- 

2 In D.C. we used the generic rate to estimate the central office costs for the entire demonstration 
and then spread these costs evenly among treatment group members. Hence, even though direct 
costs are different between the three treatments, the central office costs are the same. 
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employment Demonstration. The costs of structured JSA in New Jersey were $169 per claimant, which 
is substantially lower than the costs of SJSA in either D.C. ($286 per claimant) or Florida ($241 per 
claimant). A key factor contributing to this difference was inflation of staff compensation costs between 
1986-87, when the New Jersey demonstration operated, and 1995-96, when the D.C. and Florida 
demonstrations operated. Using the increase in the employment cost index for state and local workers, we 
can translate the New Jersey demonstration costs into 1995-96 dollars. The cost of the New Jersey 
demonstration in 1995-96 dollars was $246 per claimant, which is very close to the Florida cost estimate 
and somewhat less than the D.C. estimate. 3 This suggests that inflation of compensation, costs accounts 
for most of the difference between the New Jersey cost estimates and the D.C. and Florida cost estimates. 

Several other factors may contribute to structured JSA costs being higher in Florida and D.C. than in 
New Jersey. For example, D.C. staff tend be more highly paid than New Jersey staff, which at least partly 
reflects the high cost of living in D.C. Another factor potentially contributing to higher costs in Florida 
relative to New Jersey is the relatively small number of participants in group services in Florida, which 
tends to increase costs per claimant. 

C. SENSITIVITY OF THE COST ESTIMATES TO ALTERNATIVE ASSUMPTIONS 

The cost estimates described above are based on applying a series of assumptions to the actual 
demonstration charges. There is considerable uncertainty associated with the use of these estimates 



3 The inflation of the New Jersey costs is based on the seasonally adjusted employment cost 
index based on total compensation for state and local workers. We use the values for December 
1986 and December 1995, which are 89.0 and 129.3. 
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to evaluate the costs of an ongoing JSA program. First, some of the assumptions, while necessary to 
generate the estimates, may not accurately reflect reality. For example, we assumed that per- claimant 
costs in D.C. were similar in the pilot phase and the lull demonstration phase, but we have some basis to 
doubt that assumption. Second, some of the demonstration charges are likely to be associated specifically 
with the establishment and monitoring of the demonstration and -may not accurately reflect costs that would 
be encountered in an ongoing program. For example, time spent by central office staff in training might 
be shorter or nonexistent in an ongoing program. 

To address this uncertainty and evaluate the effects of different factors on our cost estimates, we 
generated alternative sets of estimates based on changes in the assumptions underlying the calculations. 
The new estimates, which are presented in Table IV.3, demonstrate the degree to which the cost estimates 
are sensitive to alternative assumptions. 

The first set of new estimates (alternative estimates 1 in Table IV.3) are based on an alternative 
treatment of the costs of the D.C. pilot phase. In calculating the baseline estimate of costs per 
claimant, we simply divided the combined pilot and full demonstration costs by the number of 
claimants in the pilot and full phases. The calculation implicitly assumes that costs per claimant were 
the same in the pilot phase as in the full demonstration. But, since the office was fully staffed during 
the pilot phase even though fewer claimants were served, an alternative assumption would be to 
assume that costs per week were the same between the two phases, despite the lower level of activity in 
the pilot phase. Accordingly, we generated alternative estimates of the full phase costs by dividing 
four-fifths of the total demonstration charges by the number of claimants in the full 
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phase. 4 This treatment of pilot costs reduces the estimate of the D.C. demonstration costs per claimant by 
about 1 1 percent, as shown in Table IV.3. 

The estimates presented in Table IV.2 suggest that the central office costs represent a substantial 
proportion of the demonstration costs in Florida and D.C. But in an ongoing program, the central office 
staff are likely to spend less time managing the program in a typical year than was spent managing a new 
demonstration. To investigate the impact of accounting for the possibility of lower central office costs, 
we generated alternative cost estimates (alternative estimates 2 in Table IV.3) based on a 50 percent 
reduction in the estimated central office costs. 

The impact of reducing the central office costs is to generate a modest reduction in the operational 
demonstration costs. The impact of this reduction is to reduce the total costs of the Florida treatments by 
7.5 percent for SJSA, 18.6 percent for IJSA, and 17.5 percent for USA+. A similar adjustment in the D.C. 
cost figures generates similar findings-overall operational cost estimates are reduced by 6 to 9 percent. 

Alternative estimates 3 and 4 presented in Table IV.3 are intended to gauge the extent to which the 
Florida costs could be reduced in an ongoing program by increasing the scale of local operations. This 
issue arises because the group services, including orientation, testing, and the job search workshop, tended 
to include small numbers of claimants, so that these group services were relatively costly on a per-claimant 
basis in Florida. We used two methods to estimate how per-claimant costs would be reduced by increasing 
the scale of the group services. Our first method was to assume attendance at group services was double 
the actual attendance and then recalculate the cost estimates based on this assumption. Table IV.3 shows 
that this assumption generates large reductions in the cost estimates. Since SJSA was the most heavily 

4 We used four-fifths of the demonstration charges because the pilot phase operated for one 
calendar quarter compared with four calendar quarters of the full demonstration phase. Hence, if 
costs per week were similar between the phases, the costs associated with the full phase would 
represent four-fifths of the total costs. 
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dependent on group services, the largest cost reduction, which is equal to 45 percent of the baseline 
estimate, occurs for this treatment. Doubling group service attendance generates smaller but still 
substantial reductions in IJSA and IJSA+ costs of 24 percent. 

A second method we used to address this issue is to exclude the smallest Florida offices from the cost 
calculations. When we examined demonstration costs by office, we found that per-claimant costs varied 
widely. Local demonstration costs ranged from $86 per claimant to $253 per claimant, depending on the 
office. But we found no clear relationship between the per-claimant costs in a given office and the number 
of demonstration participants in that office. Hence, excluding the small offices from the cost calculations 
had only a modest effect on estimated per-claimant costs. As shown in Table IV.3, excluding the three 
smallest demonstration offices from the calculations yielded cost figures that were only about 4 percent 
lower than the baseline estimates. 5 

Our final set of estimates is based on combining alternative estimates. For D.G., we combined the 
change in the pilot cost treatment and the 50-percent reduction in central office costs to yield cost estimates 
that are roughly 20 percent less than the baseline estimates. For Florida, we decided that achieving savings 
by doubling the scale of the Florida offices is probably unrealistic, especially given that we did not find 
a strong relationship between office caseloads and per claimant costs. Hence, we ignored alternative 3 and 
instead, combined alternatives 2 and 4~the estimates based on a 50 percent reduction in central office costs 
and those that exclude the smallest offices from the cost calculations. Combining these alternatives yields 
Florida cost estimates that are 12 to 23 percent less than the baseline estimates. 

We consider the combined alternative estimates to represent a rough lower-bound of the costs of an 
ongoing program. At the same time, we consider the baseline estimates to be upper-bound estimates, since 
it is hard to envision factors that would make an ongoing program more expensive than the demonstration. 

5 In these calculations we excluded the Pensacola, St. Augustine, and Clearwater offices. These 
three offices had the fewest claimants assigned to each of the JSA group services. 



The costs of an ongoing program therefore are probably between the baseline estimates and the estimates 
on the final line in Table IV.3. 
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V. IMPACTS ON UNEMPLOYMENT INSURANCE RECEIPT 



It was expected that by encouraging workers to search more effectively and aggressively for a new 
job, the JSA demonstration services would speed re-employment and reduce the UI benefits claimed 
by the demonstration participants. Our analysis of the impacts on UI outcomes was intended to 
determine whether the JSA treatments reduced the level of UI receipt and, if so, the extent of the 
reduction. Our approach to estimating the impacts on UI receipt was to compare UI receipt among 
claimants in the treatment groups with UI receipt among claimants in the control group. Because 
claimants were randomly assigned to the treatment and control groups, the differences in outcomes 
between the treatment and control groups provide unbiased estimates of the treatment impacts. 

In this chapter, we address three types of outcomes related to UI receipt. In Section A, we present 
estimated impacts of the JSA treatments on broad measures of average UI benefit receipt in the initial 
benefit year (year 1) and in the following year (year 2), including weeks of UI benefits, dollars of UI 
benefits, and whether claimants exhausted their benefit entitlements. In addition to the impacts on 
average UI receipt, we also estimated the impact of the treatments on the overall distribution of weeks 
on UI in our analysis of UI exit rates. This analysis helped us to identify those weeks during a UI spell 
in which the impacts of the JSA treatments typically occur. Finally, we expected that the treatments 
might have greater impacts on particular subgroups of claimants, such as those with the highest 
predicted probabilities of benefit exhaustion. As documented in Section C, we estimated the impacts 
of the treatments for various claimant subgroups. 

The estimates presented in this chapter demonstrate that the JSA treatments reduced UI receipt 
significantly over the initial benefit year. The largest impact occurred in D.C. for the SJSA treatment, 
which reduced average UI receipt by more than a week, or by $182 per claimant. The other five JSA 
treatments across the two states had more modest impacts, reducing average UI receipt by about half 
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a week, or by $100 or less. Most of the treatments also significantly reduced the proportion of claimants 
who exhausted their benefits in the initial benefit year. In the longer run, however, none of the 
treatments had an impact on new claims or on the amount of UI benefits received beyond the initial 
benefit year. 

The investigation of the UI exit rates revealed two important findings. First, at least some of the 
treatments reduced UI spells in the initial benefit year among claimants who, in the absence of the 
treatments,, would have had relatively long UI spells and were likely to exhaust their benefits. Second, 
the impacts on UI exit rates occurred early in the UI spells, around the time claimants were notified of 
the JSA service requirements or would have been scheduled to participate in services. This finding 
suggests that much of the impact on UI receipt results from an immediate response by claimants to the 
services. 

Our subgroup analysis of impacts in the initial benefit year was designed to provide information 
about the extent to which demonstration services were targeted to those most likely to be affected by 
those services, and about the extent to which impacts on UI outcomes vary across different claimant 
subpopulations. The results suggest that even if D.C. and Florida had selected even more restrictive 
eligibility criteria, i.e. higher probability thresholds that excluded either the bottom quarter or half of 
those actually deemed eligible, the average impact of being assigned to one of the JSA treatments would 
have been largely unaffected. Furthermore, in D.C., the impacts of these treatments on UI benefits were 
particularly large for young people (all three treatments) and for whites (USA and USA+). In Florida, 
the impacts of the treatments were particularly large for women (USA and USA+) and those previously 
employed in trade industries (SJSA). 

A. UNEMPLOYMENT INSURANCE RECEIPT 

The primary outcome of interest in evaluating the JSA demonstration is UI benefit receipt. For 
most of our analysis, we used three measures to examine UI receipt among the demonstration 
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participants: (1) the numbers of weeks for which each claimant was paid benefits in the benefit year, 
(2) the dollar amount of UI benefits paid to claimants in the benefit year, and (3) whether claimants 
exhausted their benefits. 

1. Average Benefit Receipt in the Control Group 

The control group provides a basis for evaluating what would have happened to the target 
population without the demonstration. Average UI receipt in the initial benefit year among the control 
groups in each state is shown in Table V.l. A comparison of UI receipt in the two states shows that the 
control group in D.C. received greater UI benefits during the initial benefit year than did the control 
group in Florida. Control group claimants in D.C. received 20.1 weeks of benefits, on average, 
compared with 15.8 weeks for claimants in Florida. Total benefit receipt in dollars was $4,236 in D.C. 
compared with $2,728 in Florida, a difference of more than 50 percent. The degree to which claimants 
exhausted their benefits was also much higher in D.C.—about 59 percent of D.C. claimants exhausted 
benefits compared with 45 percent of Florida claimants. 

The differences in the average UI measures in the two states arose primarily because average UI 
entitlements were higher in D.C. than in Florida. The average benefit entitlement in dollars for control 
group members was $5,463 in D.C. compared with $4,056 in Florida (see Chapter n, Table II.3). This 
difference in UI entitlement arose primarily from the differences in average weekly benefit amounts in 
the two states, which was caused partly by the difference in each state’s statutory maximum weekly 
benefit. In D.C., the maximum weekly benefit amount was set at $347 until January 1996, when it was 
increased to $359. This was considerably higher than Florida’s 
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ESTIMATED IMPACTS OF STRUCTURED JOB SEARCH ASSISTANCE ON UI 
OUTCOMES IN THE INITIAL BENEFIT YEAR (YEAR 1) 

(Standard Errors in Parentheses) 
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♦Statistically significant at the 90 percent confidence level in a one-tailed test. 
♦Statistically significant at the 95 percent confidence level in a one-tailed test. 
♦Statistically significant at the 99 percent confidence level in a one-tailed test. 



maximum weekly benefit amount which was set at $250 during the demonstration. In terms of actual 
benefit amounts, claimants in the D.C. control group received $212 per week on average, compared 
with $175 per week for those in the Florida control group. 

D.C. claimants could also, on average, receive benefits for a longer maximum period — they were 
all entitled to 26 weeks of benefits. Most Florida claimants were also entitled to 26 weeks of benefits, 
but some were entitled to as few as 10 weeks of benefits. Across all Florida control group members, 
average potential duration was 22.7 weeks. 

One implication of the cross-state differences in control group UI receipt is that D.C. might have 
had greater potential than Florida for UI benefit reductions. For example, a 5 percent reduction in the 
length of UI spells would have translated into greater UI savings in D.C. because the D.C. control group 
received more benefits than the Florida group. Even a one-week reduction in UI spells would likely 
have generated greater savings in D.C., since the average weekly benefit amount was higher in D.C. 
than in Florida. 1 

2. Estimated Impacts on UI Receipt 

We estimated the impact of the JSA treatments on UI receipt by comparing average UI receipt for 
the control group with that for the treatment groups. Since claimants were assigned randomly to these 
groups, the differences in average UI receipt are unbiased estimates of the impact of the treatments on 
average UI receipt. Alternatively, we could have presented regression estimates of the treatment 
impacts. The use of regressions would have allowed us to control for variation in personal 
characteristics in isolating the impacts on UI receipt. However, we tested this alternative and found 



'This is not necessarily the case, since claimants responding to the treatments in the two states 
may have had different average weekly benefit amounts. 
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that regression estimates were nearly identical to estimates based on treatment-control differences. We 
therefore present only the treatment-control differences of means. 

We present average UI outcomes and estimated impacts for both year 1— the initial benefit year- 
and year 2-the year following the initial benefit year. The year 2 UI outcomes include all UI receipt 
resulting from an initial claim filed within one year of the end of the initial benefit year. 



a. Structured JSA 

Our estimates demonstrate that SJSA clearly reduced average UI receipt in the D.C. demonstration. 
As shown in Table V.l, SJSA reduced UI receipt in the initial benefit year by 1.13 weeks, which 
represents 5.6 percent of the average UI weeks paid to the control group. In dollar terms, SJSA reduced 
UI receipt by $182 per claimant. SJSA also reduced the rate at which claimants exhausted their UI 
benefits— the exhaustion rate was nearly 5 percentage points lower for the treatment group than for the 
control group. All of these estimates are statistically significant at least at the 95 percent confidence 
level. 

The estimates for Florida provide somewhat weaker evidence that SJSA reduced UI receipt in that 
state. In Florida, the estimated impact of SJSA on the length of the average UI spell was only -0.41 
weeks, which represents 2.6 percent of the control group mean. Similarly, the estimated benefit 
reduction in dollar terms was only $39 per claimant, while the estimated impact on benefit exhaustion 
was a modest 1.8 percentage point reduction. The estimated impacts on UI weeks and benefit 
exhaustion are statistically significant at the 95 and 90 percent confidence levels, respectively. 

Although the estimated impacts in Florida are small relative to those in D.C., they are broadly 
consistent with the findings from the New Jersey UI Re-employment Demonstration. In New Jersey, 
we estimated that a structured JSA package reduced the duration of benefit receipt by 0.47 weeks 
(compared with 0.41 weeks in Florida). The estimated reductions in dollars of benefits and probability 
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of exhaustion were somewhat higher in New Jersey ($87 and 2.8 percentage points, respectively) than 
in Florida, although estimates from both states suggest modest reductions in UI receipt. 

The differences in the impacts of SJSA on UI in D.C. and in Florida were probably due to a variety 
of factors, but one potential contributor was D.C.’s relative aggressiveness in enforcing the 
demonstration participation requirements. 2 As explained in previous reports on the JSA demonstration, 
c laimants in D.C. who did not comply with the JSA service requirements (no-shows) were not sent a 
benefit check. The no-shows were then expected to report to a UI claims examiner in the demonstration 
office and reschedule their service before they could receive their checks. In contrast, Florida did not 
automatically interrupt the payment of benefits to no-shows. Florida claimants who missed a service 
could continue to receive their benefits and reschedule missed services over the telephone rather than 
reporting to the demonstration office in person. Only after a Florida claimant failed to show for the 
same service three times was the next benefit claim rejected. 

The more rigorous enforcement of demonstration participation requirements in D.C. could have 
had two effects. First, enforcing the D.C. requirements may have increased the service participation 
rate, which could have increased the effectiveness of the job search activities among the target group, 
thereby generating more rapid re-employment. Findings presented in Chapter IV confirm that among 
the SJSA claimants, the orientation and job search workshop attendance rates were higher in D.C. than 
in Florida. Second, enforcing the D.C. requirements may have caused claims to be scrutinized more 
closely so that the D.C. no-shows were denied benefits more often than Florida no-shows. 3 Either of 
these potential effects would have reduced UI receipt— in one case because claimants would have gone 




2 Another potential contributor is the higher control group receipt in D.C. compared with Florida, 
which was mentioned in the previous section. The higher control group receipt in D.C. may have 
provided greater potential for UI benefit reductions in D.C. than in Florida. 

3 We examine benefit denial rates in Chapter VI. 
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back to work more quickly, and in the other case because claimants would have been denied benefits 
more readily. 

In both of the JSA demonstration states, the estimated impacts on UI dollars were smaller than 
migh t have been expected given the impacts on weeks. However, this difference can be explained by 
results suggesting that the impacts of JSA treatments on weeks of UI receipt were larger for claimants 
with small weekly benefit amounts. If the impact of each JSA treatment were independent of the 
weekly benefit amount, 4 then we should be able to impute the average impact on UI dollars by 
multiplying the average impact on weeks by the average weekly benefit amount. The imputed impacts 
of SJSA based on this method are a $240 benefit reduction in D.C. and a $72 reduction in Florida. 
These imputed impacts exceed the direct estimates of the impacts on UI dollars that are shown in Table 
V.l ($1.82 in D.C. and $39 in Florida). This discrepancy suggests that the impact of SJSA on weeks 
paid was larger for claimants with low monthly benefit amounts. According to our calculations in D.C., 
SJSA reduced weeks paid by two weeks for claimants in the lowest quartile of the weekly benefit 
amount, by slightly more than one week for claimants in the middle two quartiles, and by nothing for 
claimants in the top quartile. 5 As a result, the reduction in the number of weeks for which benefits 



4 In the simplest case, suppose that the impact of each treatment is identical for all claimants, 
regardless of their weekly benefit amounts. 

5 Allowing the impact to vary with the weekly benefit amount , we again imputed the impact of 
SJSA on UI dollars from the estimated impacts on weeks paid. For each quartile of the weekly 
benefit amount, we multiplied the impact on weeks paid by the average weekly benefit amount in 
the quartile to impute the impact on UI dollars for that quartile. We then averaged the imputed 
impacts across all four quartiles to impute an average reduction in UI payments of $175. This 
imputed impact is very close to the direct estimate of -$ 1 82. 
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were paid led to a reduction in UI payments that was smaller than it would have been had the impact 
been unrelated to the weekly benefit amount. 

The SJSA treatment did not have an effect on UI receipt beyond the initial benefit year. Table 
V.2 presents the estimated impacts of SJSA on measures of UI receipt in year 2. SJSA had no 
statistically significant impact on any of the year 2 UI measures in either D.C. or Florida, and the point 
estimates are consistently small. For example, in D.C., the rate of new claim filing in year 2 was nearly 
identical for the SJSA and the control groups-17.1 percent for the control group and 17.0 percent for 
the SJSA group— implying that the SJSA treatment had no substantial impact on new claims. The rate 
at which new claims were filed in year 2 was somewhat lower in Florida than in D.C., but the rates were 
similar for the Florida SJSA and control groups-around 14 percent for both groups— further confirming 
the SJSA had no impact on new UI claims. The rates of average weeks of UI receipt, dollars of UI 
receipt, and benefit exhaustion were also similar across the SJSA and control groups in both states. 

Our finding that SJSA had no impact on UI receipt in year 2 differs front findings from the New 
Jersey UI Re-employment Demonstration (Corson and Haimson 1996). In that demonstration, 
researchers found that a structured JSA package significantly reduced UI receipt in the second year 
after the initial claim as well as in the initial benefit year. Based on this finding, the researchers 
speculated that the JSA package induced participants to find jobs that were, on average, more stable 
than jobs found by the control group. If the SJSA treatment in the JSA Demonstration generated 
similar long-run job stability in D.C. or Florida, it is apparently not reflected in reduced year 2 UI 
receipt Although this finding is not consistent with the findings from the New Jersey 
demonstration, it is consistent with our expectations— we did not expect that SJSA would induce 
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ESTIMATED IMPACTS OF STRUCTURED JOB SEARCH 
ASSISTANCE ON UI OUTCOMES IN YEAR 2 
(Standard Errors in Parentheses) 
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claimants to take jobs that differed substantially from those held by the control group. We re-examine 
the issue of job stability in the Chapter VII, in which we present estimated impacts on employment and 
earnings in year 2. 

b. Individualized JSA 

The impacts of the two individualized JSA treatments (USA and IJSA+) on UI receipt were 
estimated just as the impact of SJSA was estimated~by comparing average outcomes of the treatment 
groups with those of the control group. In this section, we present estimates of the impacts of USA and 
of USA+. However, our analysis of service participation in Chapter IV suggests that there was little 
distinction between the two treatments with respect to the actual services received, and we therefore 
expected the treatments to have similar impacts on UI receipt. The estimates presented in this section 
are consistent with this expectation. 

Across the two states, IJSA and IJSA+ appear to have had modest impacts on UI receipt. In D.C., 
the treatments reduced average UI receipt by about half a week—0.47 weeks for USA and 0.61 weeks 
for USA+ (Table V.3). Both of these impacts are statistically significant at the 95 percent confidence 
level The USA and USA+ groups also received less than the control group in UI dollars, but the 
differences are small and not statistically significant. IJSA+ reduced the rate of benefit exhaustion by 
an estimated 3.9 percentage points. The corresponding estimate for USA shows that USA also reduced 
UI exhaustion. The point estimate for the USA impact (2.4 percentage points) is smaller than the USA+ 
estimate, but the difference is not statistically significant 

The individualized treatments had similarly modest impacts on UI receipt in Florida. As shown 
in Table V.3, both USA and IJSA+ reduced UI receipt by about half a week, and these estimates are 
statistically significant at least at the 95 percent confidence level. USA also reduced UI dollars receipt 
by $100 and UI exhaustion by 2.4 percentage points. USA+ had broadly similar impacts. 
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ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH ASSISTANCE TREATMENTS ON UI 
OUTCOMES IN THE INITIAL BENEFIT YEAR (YEAR 1) 

(Standard Errors in Parentheses) 
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reducing UI dollars by an estimated $73 and benefit exhaustion by 2.8 percentage points. It is 

somewhat surprising that the point estimates for IJSA and IJSA+ in Florida are larger than the SJSA 
estimate, although the differences are not statistically significant. Our expectation was that SJSA 
impacts would be larger than the USA and USA+ impacts, which was the case in D.C. 

Neither of the individualized treatments had impacts on UI receipt in the longer run. Like SJSA 
impacts, the USA and IJSA+ impacts on UI receipt in year 2 were typically small and statistically 
insignificant. For example, Table V.4 shows that in D.C., the rate at which new claims were filed was 
slightly lower for the USA and IJSA+ groups than for the control group, but neither of the differences 
is significantly different from zero. In Florida, the rate at which new claims were filed was slightly 
higher for the USA and USA+ groups, but again the differences are not statistically significant. Given 
die modest impacts of USA and IJSA+ on UI receipt in year 1, we are not surprised to find no 
substantial impacts in year 2. 

If we compare the estimated USA and USA+ impacts with the estimated SJSA impacts in the two 
states, we cannot definitively conclude that one service strategy was more effective than the others in 
reducing the duration of UI spells. In D.C., the SJSA impact in year 1 was about half a week larger than 
the average USA and USA+ impacts, and the difference is statistically significant. But in Florida, the 
SJSA impact in year 1 was nearly identical to the USA and USA+ impacts. In year 2, none of the 
treatments had a significant impact on UI receipt in either state. 

The differences (or lack of differences) between the impact estimates for USA and IJSA+ are not 
indicative of the impact of training on demonstration claimants. We noted in Chapter IV that the rate 
of training participation was no higher in the USA+ group than in the USA group. Given this finding, 
it is not surprising that the impacts on the two groups were similar. Any differences in 
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ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH ASSISTANCE TREATMENTS ON U 1 OUTCOMES IN YEAR 2 

(Standard Errors in Parentheses) 
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outcomes between the two groups therefore cannot be attributed to training and are more likely due to 
sampling variation. 

B. IMPACTS ON Cl EXIT RATES 

The previous section provides estimates of the impact of the JSA treatments on mean UI outcomes. 
Additional information is provided by examining the impact of the JSA treatments on the entire 
distribution of weeks on UI. In this section, we present for each state the weekly UI exit rates for the 
control group and the JSA treatment groups, as well as impact estimates based on the difference 
between these weekly rates. 

1. Structured JSA 

The weekly cumulative exit rates for the control group and SJSA group for D.C. are shown in 
Figure V.l. These cumulative rates show the proportion of claimants in each group whose UI spell was 
equal to or less than the week shown in the figure. 6 The cumulative exit rates for both groups increased 
steadily over time, demonstrating that D.C. claimants stopped receiving UI at a relatively steady weekly 
rate. Approximately 40 percent or more of claimants assigned to either group exited UI prior to week 
26 of their spell, which is when most claimants would have exhausted their benefits. 



6 The estimate of the cumulative UI exit rate at week j is based on the following expression: 

Hj = (1-Sj); ' 

where Sj = ? i=lj (l-l}) and is the conditional exit rate in week j. The conditional exit rate is equal to the 
number of claimants who voluntarily end their UI spell during the week divided by the number of 
claimants still receiving UI at the be ginnin g of the week. The term Sj is the Kaplan-Meier estimate of the 

survival rate— an estimate of the proportion of the original sample that has not yet exited UI. 
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FIGURE V.l 



IMPACT OF STRUCTURED JSA ON 
CUMULATIVE Ul EXIT RATES, 
DC 
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SJSA increased the rate at which D.C. claimants exited UI throughout the entire potential UI spell. 
The impact of SJSA is represented by the difference between the exit rates for the SJSA and control 
groups. This difference, which is shown in the lower panel of Figure V.l, reaches a maximum of 
slightly greater than 6 percentage points in week 6 and then hovers between 4 and 6 percentage points 
for the remainder of the UI spell. We also conducted significance tests of these differences at five-week 
intervals, which are shown in Table V.5. At each interval the difference in exit rates between the 
control group and SJSA group is significant at the 99 percent confidence level. 

In addition to being statistically significant, the differences shown in Figure V.l are substantial. 
Furthermore, the timing of the impacts suggests that SJSA affected claimants who would otherwise 
have had long-term UI spells. At the five-week mark, the cumulative exit rate for the SJSA group was 
17.7 percent, which was more than 50 percent higher than the 11.6 percent rate for the control group. 
The absolute magnitude of this difference then remained relatively steady over time, even though the 
SJSA services were received early in the UI spell. The persistence of this difference suggests that some 
claimants whose UI receipt was reduced as a function of their response to SJSA would have exhausted 
their benefits without the treatment. 7 This evidence supports the findings presented in Table V.l, which 
shows that SJSA reduced the rate of benefit exhaustion in D.C. by 5 percentage points. This finding 
is also consistent with the New Jersey UI Re-employment Demonstration, in which a similar SJSA 
treatment significantly reduced the rate of benefit exhaustion. 



’Since the SJSA-control difference persists until the point of exhaustion, this implies that some 
members of the SJSA group who exited UI early in response to SJSA have counterparts in the 
control group who did not exit UI and therefore exhausted their benefits. We can therefore conclude 
that some of the early exiters in the SJSA group would have exhausted their benefits in the absence 
of SJSA services. 
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TABLE V.5 



ESTIMATED IMPACTS OF STRUCTURED JOB SEARCH 
ASSISTANCE ON UI EXIT RATES 



Length of 
UI Spell, in 
Weeks 


District of Columbia 




Florida 




Cumulative Exit Rates (%) 


> 


Cumulative Exit Rates (%) 




Control 

Group 


SJSA 

Group 


Estimated 

Impact 


Control 

Group 


SJSA 

Group 


Estimated 

Impact 


5 


11.6 


17.7 


6.1*** 


17.5 


21.0 


3.5*** 


10 


20.1 


~ 25.3 


5.1*** 


30.4 


34.9 


4.5*** 


15 


27.9 


32.6 


4 . 6 *** 


42.7 


44.1 


1.3 


20 


32.4 


37.4 


5.0 ♦♦♦ 


50.5 


52.4 


1.9* 


25 


39.9 


44.7 


4.8*** 


58.5 


59.5 


1.0 



SOURCE: State UI claims records. 

NOTE: The impact estimates are based on differences in exit rates between the control group and the 

SJSA group. 

♦Statistically significant at the 90 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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As expected, SJSA had a smaller impact on UI exit rates in Florida than in D.C. The top panel in 
Figure V.2 shows that, in Florida, about 60 percent of the claimants in both the control group and SJSA 
groups exited UI by week 26. However, the rate for the SJSA group was higher throughout the 
observation period. The difference between the groups, which is shown in the lower panel of Figure 
V.2, reaches a maximum in week 8 of 5 percentage points, which is lower than the maximum of more 
than 6 percentage points in D.C. Furthermore, the difference between the two groups in Florida slowly 
becomes smaller after reaching a maximum, so that it hovers between 1 and 2 percentage points 
beginning in week 14. This decay in the impact of SJSA on the exit rate suggests that the Florida 
claimants whose UI spells were shortened as a function of their response to the SJSA treatment were 
unlikely to have had long UI spells even without the demonstration. 8 This finding is consistent with 
the estimated impact of SJSA on benefit exhaustion in Florida, which was relatively small (a 1.8 
percentage point reduction). 

Significance tests of the differences at five-week intervals support these findings. As shown in 
Table V.5, the estimated impact of SJSA on the cumulative UI exit rate in Florida is substantial and 
significant at weeks 5 and 10. However, this impact decays over time, and at weeks 15 and 25 the 
estimated impacts are not statistically significant. This pattern differs from the persistent significance 
of the D.C. impacts shown in the table. 

One finding that is the same for the two states is that the impact of SJSA on UI exit rates first 
appears early in the UI spells. In both states, a substantial proportion of the maximum impact occurred 
in the first three or four weeks of the UI spell. This impact occurred at approximately the 



8 The decay of the SJSA-control difference suggests that while some SJSA claimants exited UI 
earlier than their counterparts in the control group, the control group counterparts subsequently 
exited UI by week 15, too early to exhaust their UI benefits. Hence, we conclude that the early 

exiters in the SJSA group would still have exited by week 15 in the absence of SJSA. 

101 



139 



FIGURE V.2 



IMPACT OF STRUCTURED JSA ON 
CUMULATIVE Ul EXIT RATES, 
FLORIDA 
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same time that claimants were being notified of the JSA services and beginning to participate in them. 
Other tests of mandatory services provided to UI claimants have yielded similar findings. 9 

The timing of the impacts on UI exit rates implies that much of the impact of SJSA is due to an 
immediate response to being notified of the services and to participating in them. Claimants assigned 
to SJSA may have immediately conducted effective job searches using the information they received 
and techniques they learned through the various services. Similarly, claimants may simply have been 
encouraged by the JSA services to go out and find a job immediately. Finally, claimants may have 
returned to work or even simply stopped claiming benefits to avoid the JSA participation requirements. 
Our findings do not allow us to distinguish between these explanations. 

2. Individualized JSA 

The timing of the impacts of both individualized JSA treatments was similar to the timing of the 
SJSA impacts. All treatments achieved their maximum impacts early in what could have been much 
longer UI spells. In D.C., both individualized treatments achieved their maximum impact in week 6 
(Figures V.3 and V.4), while in Florida, both achieved their maximum in week 8 (Figures V.5 and V.6). 

However, the impact estimates for USA and HSA+ in both states tend to be modest and are often 
not statistically significant. In D.C., the estimated USA and HSA+ impacts, which are shown in Table 
V.6, are clearly smaller than the estimated SJSA impacts shown in Table V.5. For example, the 
maximum estimated impacts of the IJSA and USA+ treatments in D.C. were 3.2 and 4.2 



9 Corson and Decker (1989) showed that in the New Jersey UI Re-employment Demonstration 
a significant part of the impact of JSA on UI exit rates occurred at the time that claimants were 
notified of services or were required to participate in services. Johnson and Klepinger (1994) 
detected a similar effect of mandatory services in the Washington Alternative Work Search 
Experiment. 
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FIGURE V.3 



IMPACT OF INDIVIDUALIZED JSA ON 
CUMULATIVE Ul EXIT RATES, 

DC 
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FIGURE V.4 



IMPACT OF INDIVIDUALIZED JSA ON 
CUMULATIVE Ul EXIT RATES, 
FLORIDA 
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FIGURE V.5 



IMPACT OF INDIVIDUALIZED JSA WITH TRAINING ON 
CUMULATIVE Ul EXIT RATES, 

DC 
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FIGURE V.6 



IMPACT OF INDIVIDUALIZED JSA WITH TRAINING ON 
CUMULATIVE Ul EXIT RATES, 

FLORIDA 
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TABLE V.6 



ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH 
ASSISTANCE ON CUMULATIVE UI EXIT RATES 







USA 




USA + 




Length of 
UI Spell, 
in Weeks 


ControlGroup 
Exit Rate 
(Percent) 


Treatment Group 
Exit Rate 
(Percent) 


Estimated 

Impact 


Treatment Group 
Exit Rate 
(Percent) 


Estimated 

Impact 




MliliS 


wNWINNrMr 


»f Columbia 


,V-'J 


5 


11.6 


14.8 


3 . 2 *** 


15.8 


4.2*** 


10 


20.1 


22.8 


2 . 1 ** 


23.1 


3.0*** 


15 


27.9 


29.3 


1.4 


30.6 


2.7** 


20 


32.4 


34.2 


1.8 


35.1 


2.7** 


25 


39.9 


41.9 


2.0* 


42.7 


2.8** 


>< &•. -Six .J 




- * LOR ! DA ■ ■ • ' ■’ * 




5 


17.5 


20.2 


2 . 1 *** 


20.2 


2.8*** 


10 


30.4 


33.9 


3.5*** 


33.7 


3.2*** 


15 


42.7 


44.3 


1.6 


44.6 


1.8* 


20 


50.5 


52.6 


2.1* 


52.6 


2.1* 


25 


58.5 


61.4 


2.9** 


61.2 


2.7** 



SOURCE: State UI claims records. 

NOTE: The impact estimates are based on the differences in exit rates between the control group 

and the IJSA and USA + groups. 

♦Statistically significant at the 90 percent confidence level in a one-tailed test. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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percentage points, compared with 6.2 percentage points for SJSA. Among the estimated impacts at 
five-week intervals shown in Table V.6, the impacts in weeks 5, 10, and 25 are significant in both states 
and across both treatments. 

Although the impacts are modest, they are relatively persistent. They decay only partially and 
slowly after the maximum is achieved. Even out as far as week 25, the four treatment impacts across 
the two states are statistically significant at least at the 90 percent confidence level. The relative 
persistence of the impacts is especially apparent in Florida, where the IJSA and USA+ impacts decayed 
much less than the SJSA impact. The USA and USA+ treatments in Florida increased the cumulative 
exit rate as of week 25 by an estimated 2.9 and 2.7 percentage points, compared with a 1.0 percentage 
point increase for SJSA. In Florida, the lack of decay in the impacts of the individualized treatments 
on UI exit rates probably accounts for the large impacts of these treatments on UI weeks relative to 
SJSA. 

C IMPACTS ON UI RECEIPT BY CLAIMANT SUBGROUP 

Although we have presented estimates of the average impacts on all claimants targeted for JSA 
treatments, these treatments probably had different impacts on different types of claimants. Eligible 
claimants differ in basic demographic characteristics--such as gender, age, race, education-in pre-UI 
employment experience, and in the predicted probability of exhausting their UI benefits. If the impacts 
of the treatments varied along these dimensions, and we can use these dimensions to define subgroups 
of claimants, then we can measure how the impacts on UI receipt varied across subgroups. We 
performed this analysis only for the initial benefit year because our estimates of the average impacts in 
the following calendar year suggest that there are probably no long-run impacts. We also focused on 
one particular UI outcome, dollars of UI payments, but the results are similar for the other outcomes. 

We measured impacts separately for each subgroup for two reasons. First, it would be useful to 
know how the average impacts in each state would have been different had the eligibility criterion been 



stricter, that is, had the minimum predicted exhaustion probability been set higher. This analysis could 
prove useful if the states consider reducing the number of claimants required to participate in 
employment services by raising the minimum exhaustion probability necessary for eligibility . 7 This 
subgroup analysis involves estimating separately the treatments impacts for three intervals of the 
predicted exhaustion probability. The results of this analysis reveal whether the JSA impacts varied 
according to the probability of exhaustion. 

The second reason for measuring the impacts on subgroups separately is that while it may be 
difficult for the states to target mandatory services on any basis other than the predicted exhaustion 
probability, it may be useful to know the extent to which the treatment impacts varied according to basic 
demographic characteristics and pre-UI employment experiences. Suppose there was evidence that 
none of the treatments was effective in reducing the total amount of UI benefits paid to claimants with 
a college education. If states wanted to restrict services to those likely to benefit from them, then they 
might exclude college graduates from the requirement to participate in employment services. 

Even if states are prohibited from excluding certain subgroups from the eligible population (or 
choose not to do so), estimates of subgroup impacts may be useful in predicting the impacts of similar 
programs on different populations in other states. If, for example, we learn that the treatments in D.C. 
and Florida are especially effective in reducing UI benefits among those previously employed in service 
sector jobs, then we can use the industrial mix of employment in other states to predict how effective 
the three treatments would be in those states. 

For all variables used to define subgroups except for the probability of exhaustion , 10 the subgroup 
analysis controlled for all of the other variables that define subgroups for a particular set of subgroups. 



10 Since the probability of exhaustion is a function of the other variables used to define 
subgroups, according to the profiling model, controlling for these other variables when computing 
separate impacts for different ranges on the probability of exhaustion would not make sense. 



For example, when computing the impacts separately for men and women, all of the other 
characteristics used in the subgroup analysis were held constant to ensure that impacts related to 
differences in gender cannot be attributed to other variables that are both related to gender and included 
in the subgroup analysis (such as occupation). 

Our analysis succeeded in identifying many subgroups with significant treatment impacts. In many 
instances, the impacts also appear to have differed greatly across subgroups. However, the samples 
were often too small to provide compelling statistical evidence that the apparently large and apparently 
small impacts were significantly different from the average impact Furthermore, the variations in 
impacts by subgroup show very different patterns in D.C. and Florida. Given these findings, it is 
difficult to draw any strong conclusions from the subgroup analysis. In the remainder of this section, 
we describe the findings from this analysis, highlighting the cases in which the differences between 
subgroups are statistically significant. Since the patterns of subgroup estimates differ so greatly 
between the two states, we discuss the findings for each state separately. 

1. District of Columbia 

Overall, the results for D.C., presented in Table V.7, suggest that the impacts of the treatments were 
larger for whites and younger claimants, and smaller for claimants previously employed in technical 
and managerial occupations. With respect to variation in impacts by race, the strongest evidence for 
differences appears in the impacts of USA and USA+ on total UI benefits. The impacts 
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TABLE V.7 



SUBGROUP IMPACTS IN THE INITIAL BENEFIT YEAR, 
DISTRICT OF COLUMBIA: TOTAL BENEFITS 



Control 

Subgroups Mean 



Average Impacts 3 



Estimated Probability of 
Exhaustion 11 



1 st Quartile (lowest) 


4,294 


2 nd Quartile 


4,079 


3 rd Quartile 


4,368 


4 th Quartile (highest) 


4,202 


Gender 


Male 


4,251 


Female 


4,223 


Age 


Under 35 


4,225 


35 to 44 


4,350 


45 and older 


4,126 


Race/Ethnicity 




White 


4,187 


Black 


4,317 


Hispanic 


3,971 


Other 


3,900 


Education 


No High School Diploma 


3,974 


High School Diploma Only 


4,238 


College Degree 


4,402 


Missing 


4,095 


Industry at Previous Job 


Wholesale and Retail Trade 
Finance, Insurance, and 


4,144 


Real Estate 


4,723 


Services 


4,098 


Public Administration 


4,445 


Other 


4,404 



O 




Impacts 



Sample 



SJSA 


IJSA 


IJSA+ 


Size 


-185* 


-52 


-56 


8,071 



-224 


-110 


-30 


2,017 


-139 


-115 


-65 


2,018 


-42 


103 


-25 


2,018 


-335* 


-108 


-18 


2,018 



-159 


-64 


-53 


J 3,673 


-207* 


-41 


-58 


4,398 



-348* 


-259*# 


-187* 


3,398 


-164 


99 


59 


2,448 


41 


99 


18 


2,225 



-52 


-591*# 


- 537*# 


744 


-229* 


-51 


7 


5,916 


-454 


97 


-221 


376 


69 


277 


-8 


1,035 



-325 


-119 


-18 


1,140 


-154 


-95 


-63 


4,127 


-54 


12 


-157 


2,238 


-640* 


149 


315 


566 



-367* 


-75 


-124 


1,165 


-486* 


-306 


-364 


655 


-131 


-63 


-62 


4,221 


-180 


283 


-60 


653 


-58 


-35 


169 


1,377 
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Table V.7 (Continued) 



Subgroups 


Control 

Mean 


SJSA 


Impacts 

IJSA 


IJSA+ 


Sample 

Size 


Occupation at Previous Job 


Technical and Managerial 


4,403 


273*# 


-154 


136 


1,664 


Clerical and Sales 


4,111 


-236* 


32 


-47 


4,769 


Other 


4,427 


-502 


-192 


-276 


1,638 


Tenure 


Less than 1 Year 


4,274 


-176 


-171 


-37 


2,673 


1 to 3 Years 


4,241 


-265* 


-34 


-89 


2,322 


3 to 10 Years 


4,123 


-107 


81 


2 


2,137 


10 Years or More 


4,371 


-192 


-55 


-162 


939 


Base Year Wages 


1 st Quartile (lowest) 


2,410 


-33 


-18 


-108 


2,020 


2 nd Quartile 


3,609 


-223 


-186 


-141 


2,017 


3 rd Quartile 


4,790 


-335* 


-189 


• 82 


2,017 


4 th Quartile (highest) 


6,134 


-150 


187 


-56 


2,017 


Sample Size 


2,012 


2,026 


2,022 


2,011 





Source: State UI claims records. 

“These impacts were evaluated at the mean over all eligibles for each of the subgroup variables listed in the 
table except for the probability of exhaustion (see footnote b). 

b In computing the impacts for different categories of this variable, we did not control for the other subgroup 
variables listed in this table, many of which contribute to predicting exhaustion. 

♦Statistically different from zero at the 95 percent confidence level in a one-tailed test. 

#Statistically different from the average impact at the 95 percent confidence level in a two-tailed test. 
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of USA and IJSA+ for whites are -$591 and -$537, respectively, and both impacts are significantly 
different from the average effects of these treatments on benefits for all claimants (-$52 and -$56, 
respectively). 

With respect to age, impacts of the three treatments on UI receipt tended to be largest for the 
youngest group— claimants under 35. For example, SJSA reduced average UI receipt among young 
claimants by $348, which is statistically significant at the 95 percent confidence level. Similarly, the 
IJSA and IJSA+ impacts on this groups were -$259 and -$187, although the latter impact is not 
statistically significant. In contrast, five of the six estimated impacts for the older subgroups are 
positive, implying that the treatments did not reduce average UI receipt among these five groups, and 
the one negative estimate is not statistically significant. Younger workers may have been most 
responsive to the treatments because of weaker ties to a particular industry or occupation. 

There is no strong evidence that the treatment impacts varied according to a claimant’s base year 
wages, industry at previous job, or tenure at previous job, but there is evidence that the impacts of SJSA 
varied according to the claimant’s occupation at previous job. UI receipt among claimants previously 
employed in technical or managerial occupations appears to have been unaffected, or even raised, by 
the SJSA program, in contrast to claimants in other occupations. Claimants from technical and 
managerial occupations saw an estimated increase in average UI receipt of $273 (relative to the control 
group), which is insignificantly different from zero but significantly different from the average treatment 
effects of -$185. This finding may indicate that experienced technical and managerial workers are in 
such high demand that providing them with job search assistance is unnecessary. 

Contrary to our expectations, the treatment impacts do not appear to vary much according to the 
predicted probability of benefit exhaustion. Prior to the demonstration, we expected that impacts would 
probably be largest for claimants with the highest probabilities of benefit exhaustion. For SJSA, the 
impact is largest for claimants in the highest probability quartile (-$335), but while the impact is 
significantly different than zero, it is not significantly different from the average impact for all claimants 
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(-$185). Furthermore, the next largest UI reduction occurred for claimants in the lowest quartile. For 
IJSA and USA+, there does not appear to be a clear relationship between the magnitude of the impacts 
and the probabilities of benefit exhaustion. These findings imply that making the claimant targeting 
more exclusive in D.C. would probably not have substantially influenced the average treatment impacts. 

However, when we examine the top decile of claimants with respect to their exhaustion 
probabilities (instead of the top quartile), we find particularly large estimated impacts of SJSA. The 
estimated impact of SJSA on the top decile and on the bottom 90 percent are -$626 and -$134, 
respectively. While the difference between these two estimates is significant only at the 90 percent 
level, the difference between these two groups in the estimated impacts on weeks of UI benefits 
(-2.93 weeks and -0.92 weeks) is significant at the 95 percent level. These results suggest that the 
impacts of JSA treatments may be larger than average for claimants with very high benefit exhaustion 
probabilities. • • , • . 

2. Florida 

The results for Florida overall suggest that the impacts of IJSA and IJSA+ may have been larger 
for women than for men, that the impacts of SJSA may have been larger for claimants previously 
employed in trade industries, and that the impacts of USA and USA+ for claimants previously employed 
in technical, managerial, clerical or sales occupations were particularly small (or zero). 

Unlike DC, there is no evidence in Florida that impacts varied by age group and racial group, but 
there is some evidence that USA and USA+ impacts varied according to gender. Table V.8 shows that 
USA+ reduced UI payments to women by an estimated $141 but had no impact on UI receipt among 
men. The difference by gender is also statistically significant. The impact of USA varied similarly 
according to gender, although the magnitude of the difference is somewhat smaller than for USA+, and 
the impact for females is not statistically significant 
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Although there is little evidence that the treatment impacts varied with a claimant’s previous job 
tenure, there is some evidence that the impacts varied according to the claimant’s previous industry and 
occupation. With respect to industry, SJSA, and perhaps IJSA, had larger impacts on claimants 
previously employed in trade industries than on claimants employed in other industries. For example, 
SJSA reduced UI receipt among claimants from trade industries by an estimated $240, compared with 
an average reduction of -$32 among all claimants. 

Treatment impacts also varied by occupation. The UI outcomes of claimants previously employed 
in technical, managerial, clerical, or sales occupations seem to have been virtually unaffected by the 
IJSA and USA+ treatments, unlike outcomes for claimants previously employed in other occupations, 
whose UI receipt dropped substantially. For example, the impacts of IJSA and USA+ on UI receipt 
among claimants in occupations other than technical, managerial, clerical, or sales were -$181 and - 
$220, respectively, and the impact of USA+ on these claimants is also significantly different from the 
average USA+ impact on all claimants. Previous occupation is the one dimension on which the 
subgroup impacts in Florida are similar to those in D.C. As in D.C., experienced technical and 
managerial workers in Florida may have had so many reemployment opportunities that job search 
assistance was unnecessary. In Florida, this same logic may also apply to clerical and service workers. 
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TABLE V.8 



SUBGROUP IMPACTS IN THE INITIAL BENEFIT YEAR, 
FLORIDA: TOTAL BENEFITS 



Impacts 



Control 



Subgroups 


Mean 


SJSA 


IJSA 


IJSA+ 


Sample Size 


Average Impacts 3 




-33 


-58 


-68 


12,042 


Estimated Probability of 

Exhaustion 11 

l a Quartile (lowest) 


2,337 


-57 


-42 


-10 


3,005 


2 nd Quartile 


2,379 


. -15 


-242* 


-92 


3,016 


3 rd Quartile 


2,744 


64 


-23 


49 


3,009 


4 th Quartile (highest) 


3,439 


-114 


-108 


- 227* 


3,012 


Gender 


Male 


2,665 


-56 


-9 


-6 


6,504 


Female 


2,777 


-4 


-115* 


-141* 


5,538 


Age 

Under 35 


2,466 


47 


3 


-102 


3,425 


35 to 44 


2,698 


-90 


-62 


-6 


3,424 


45 and older 


2,893 


-47 


-95 


-86 


5,193 


Race/Ethnicity 

White 


2,649 


-29 


-72 


-77 


7,401 


Black 


2,731 


28 


-14 


-33 


1,880 


Hispanic 


2,906 


-102 


-46 


-96 


2,614 


Other 


2,538 


294 


-129 


428 


147 


Education 


No High School Diploma 


2,682 


-47 


-95 


-16 


3,122 


High School Diploma Only 


2,724 


-94 


-18 


-71 


6,692 


College Degree 


2,741 


173*# 


-125 


-131 


2,228 


Industry at Previous Job 


Wholesale and Retail Trade 


2,688 


-240*# 


-203* 


-70 


2,012 


Finance, Insurance, and 
Real Estate 


2,754 


-38 


-215* 


-206 


1,281 


Services 


2,519 


95 


-7 


-62 


3,200 


Manufacturing 


2,703 


33 


-18 


86 


1,551 


Other 


2,882 


-54 


9 


-88 


3,998 
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Table V.8 (Continued) 



Subgroups 


Control 

Mean 


SJSA 


Impacts 

IJSA 


IJSA+ 


Sample Size 


Occupation at Previous Job 


Technical and Managerial 


2,748 


137 


80 


-15 


2,460 


Clerical and Sales 


2,697 


-113* 


-25 


27 


5,365 


Other 


2,722 


-29 


-181* 


-220*# 


4,217 


Tenure 


Less than 1 Year 


2,507 


-16 


-67 


9 


4,372 


1 to 3 Years 


2,692 


-57 


-114 


22 


3,005 


3 to 10 Years 


2,846 


-51 


0 


-149* 


3,464 


10 Years or More 


3,158 


23 


-51 


-335*# 


1,201 


Base Year Wages 


1 st Quartile (lowest) 


1,450 


-69 


-64 


-141 


3,019 


2 nd Quartile 


2,145 


-49 


52 


-57 


3,008 


3 rd Quartile 


3,341 


-126 


-309*# 


-196* 


3,008 


4 th Quartile (highest) 


3,928 


115 


89 


123# 


3,007 


Sample Size 


3,014 


3,032 


3,007 


2,989 





Source: State UI claims records. 

These impacts were evaluated at the mean over all eligibles for each of the subgroup variables listed in the 
table except for the probability of exhaustion (see footnote 2). 

b In computing the impacts for different categories of this variable, we did not control for the other subgroup 
variables listed in this table, many of which contribute to predicting exhaustion. 

♦Statistically different from zero at the 95 percent confidence level in a one-tailed test 

#Statistically different from the average impact at the 95 percent confidence level in a two-tailed test. 




118 



156 



There also appears to be some relationship between the size of the treatment impacts and base year 
wages in Florida. The impacts of all three treatments seem to be particularly large for claimants with 
base year wages in the second highest quartile, yet particularly small for claimants in the highest 
quartile. The impact of USA on claimants in the second highest quartile is significantly different from 
zero and from the average impact on all claimants, while the impact of USA+ is significantly different 
from zero but not from the average impact 

As in D.C., we found no strong evidence in Florida that the treatment impacts were consistently 
larger for claimants with higher predicted probabilities of benefit exhaustion. For two of the three 
treatments, SJSA and USA+, the largest estimated impact occurred for the highest probability quartile. 
However, the SJSA impact is not significantly different from zero, and the USA+ impact, while 
significantly different from zero, is not significandy different from the average impact across all 
claimants. Unlike the situation in D.C., we found no evidence in Florida that the impacts were 
particularly large for claimants in the top decile of the predicted benefit exhaustion probabilities. These 
findings, together with the findings from D.C., demonstrate that targeting the JSA treatments to 
claimants with the highest predicted benefit exhaustion probabilities would not have substantially 
affected the average treatment impacts. 
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VL IMPACTS ON UI BENEFIT DETERMINATIONS AND DENIALS 



Claimants assigned to one of the JSA treatment groups were required to participate in JSA 
services. Claimants who refused to participate in services were at risk of being denied at least part 
of their benefits. Enforcement of the JSA participation requirements was designed to occur through 
the procedures used to enforce normal UI eligibility requirements. To track JSA participation among 
claimants assigned to the demonstration, each state added an eligibility requirement based directly 
on demonstration participation and tracked it throughout the demonstration. Because the JSA 
treatments established requirements to participate in various services, we expected that the 
treatments would increase the frequency with which the states evaluated claimants’ eligibility for 
benefits. We also expected that the demonstration requirements would lead to more frequent benefit 
denials. 

In this chapter, we present our estimates of the impacts of the JSA treatments on UI benefit 
eligibility determinations and denials. We estimated these impacts by comparing the frequency of 
determinations and denials for the control group with that of the JSA treatment groups. Our findings 
demonstrate that all of the treatments increased the frequency of determinations and denials. Many 
of these impacts are not directly attributable to the enforcement of the demonstration requirements. 
Our findings show that the enforcement of the demonstration requirements increased the frequency 
of determinations and denials not directly related to the demonstration as well as those due to 
demonstration nonparticipation. It appears that the demonstration states used additional information 
gathered through the demonstration to increase enforcement of traditional UI eligibility requirements. 



A. BENEFIT DETERMINATIONS AND DENIALS AMONG THE CONTROL GROUPS 



The control groups in D.C. and Florida had generally similar experiences with respect to the 
likelihood of benefit determinations and denials. The frequencies of determinations and denials for 
the control group and for the SJSA group are shown in Table VI. 1. In each state, about 25 to 30 
percent of control group claimants had at least one nonseparation, nonmonetary benefit 
determination, with the probability of a determination being slightly higher in D.C. About 9 or 10 
percent of control group claimants had at least one denial. 

Although the likelihood of having at least one determination was broadly similar in the two 
states, the average number of determinations per claimant was considerably higher in D.C. than in 
Florida— 0.61 determinations per claimant in the D.C. control group compared with 0.41 in the 
Florida control group. The combination of these findings suggests that D.C. claimants were more 
likely than claimants in Florida to have multiple determinations. Indeed, as shown in Table VI. 1, 
12.8 percent of D.C. control group claimants had more than one determination, compared with 8.1 
percent of Florida control group claimants. 

Although the average number of determinations per claimant for the control groups differed 
from state to state, the number of denials per claimant was similar for the control groups in the two 
states— 0.12 denials per claimant in D.C. and 0.13 denials per claimant in Florida. Combining the 
determination and denial findings demonstrates that the probability of a determination resulting in 
a denial was higher in Florida than in D.C. As shown in Table VI. 1, 31 percent of determinations 
resulted in denials for the Florida control group, while only 20 percent resulted in denials in D.C. 

Our findings suggest that D.C. tended to be a bit more aggressive than Florida in conducting 
benefit determinations for the type of claimants targeted by the demonstration. At the same time. 
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TABLE VI. L 



ESTIMATED IMPACTS OF STRUCTURED JOB SEARCH 
ASSISTANCE ON NONSEPARATION, NONMONETARY 
DETERMINATIONS AND DENIALS 





District of Columbia 




Florida 




Outcome 


Control 

Group 


SJSA 

Group 


Estimated 

Impact 3 


Control 

Group 


SJSA 

Group 


Estimated 

Impact 3 


Determinations 














Percent with at Least 
One 


30.2 


66.8 


36.6*** 

(1.5) 


26.9 


31.2 


4.4*** 

(1.2) 


Percent with More Than 
One 


12.8 


36.1 


23.3*** 

(1.3) 


8.1 


11.1 


3.1*** 

(0.8) 


Average Number per 
Claimant 


0.61 


1.39 


0.78*** 

(0.05) 


0.41 


0.53 


0.12*** 

(0.03) 


Denials 














Percent with at Least 
One 


10.3 


21.1 


10.8*** 

(1.1) 


8.7 


11.6 


2.9*** 

(0.8) 


Percent with More Than 
One 


1.7 


3.1 


1.4*** 

(0.5) 


2.2 


3.0 


0.8** 

(0.4) 


Average Number per 
Claimant 


0.12 


0.25 


0.13*** 

(0.01) 


0.13 


0.17 


0.05*** 

(0.01) 


Percent of 
Determinations 
Leading to Denial 


19.9 


17.8 




31.3 


33.0 




Sample Size 


2,012 


2,026 




3,014 


3,032 





SOURCE: State UI claims records. 

Note: The impact estimates are based on differences in outcomes between the control group and the 

SJSA group. 

“Standard errors in parentheses. 

^Statistically significant at the 95 percent confidence level in a one-tailed test. 

* ^Statistically significant at the 99 percent confidence level in a one-tailed test. 
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the states denied benefits at about the same rate, so more aggressive determinations in D.C. did not 
translate into more frequent benefit denials. 

B. IMPACTS ON DETERMINATIONS AND DENIALS 

1. Structured JSA 

As expected, the probability of benefit determination in both D.C. and Florida was higher for 
the SJSA group than for the control group, but the differences varied greatly between the two states. 
The difference in determinations was much larger in D.C., where the probability of determination 
was 36.6 percentage points higher for the SJSA group than for the control group (Table VI. 1), and 
the difference is statistically significant at the 99 percent confidence level. The corresponding 
difference in Florida was considerably smaller at 4.4 percentage points, although it is still statistically 
significant 

The probability of benefit denial was also significantly higher for the SJSA groups than for the 
control groups. Again, the larger impact occurred in D.C. The probability of having at least one 
denial in D.C. was 10.8 percentage points higher for the SJSA group than for the control group. The 
corresponding difference in Florida was smaller at 2.9 percentage points. 

2. Individualized JSA 

Our findings for the impact of USA and USA+ on determinations and denials generally parallel 
our findings for SJSA. The impacts were larger in D.C., where both IJSA and USA+ increased the 
proportion of claimants having at least one benefit determination by about 29 percentage points, as 
shown in Table VI.2. These impacts represent a doubling of the determination rate for the control- 
group, and they are both statistically significant at the 99 percent confidence level. USA and USA+ 
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TABLE VI. 2 



ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH ASSISTANCE TREATMENTS 
ON NONSEPARATION, NONMONETARY DETERMINATIONS AND DENIALS, 

DISTRICT OF COLUMBIA 







IJSA 


USA + 


Outcome Measure 


Control 

Group 

Outcome 


Treatment 

Group 

Outcome 


Estimated 

Impact 


Treatment 

Group 

Outcome 


Estimated 

Impact 


Determinations 












Percent with at Least One 


30.2 


59.2 


29.0*** 

(1.5) 


58.9 


28.7*** 

(1.5) 


Percent with More Than One 


12.8 


26.0 


13.1*** 

(1.2) 


27.6 


14.8*** 

(1.2) 


Average Number per Claimant 


0.61 


1.06 


0.45*** 

(0.04) 


1.11 


0.51*** 

(0.04) 


Denials 












Percent with at Least One 


10.3 


18.4 


8.1*** 

(1.1) 


17.3 


7.0*** 

(1.1) 


Percent with More Than One 


1.7 


2.5 


0.8** 

(0.5) 


3.1 


1.4*** 

(0.5) 


Average Number per Claimant 


0.12 


0.21 


0.09*** 

(0.01) 


0.21 


0.09*** 

(0.01) 


Percent of Determinations 
Leading to Denial 


19.9 


20.1 




18.7 




Sample Size 


2,012 


2,022 




2,011 





Source: State UI claims records. 

Note: The impact estimates are based on differences in outcomes between the control group and the 

IJSA and IJSA + groups. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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also significantly increased the proportion of D.C. claimants with multiple determinations, again by 
doubling the control group rate. 

In Florida, USA and IJSA+ increased the probability of a benefit determination by about 3 
percentage points, as shown in Table VI.3. Although these impacts are much smaller than in D.C., 
they are still statistically significant at the 95 percent confidence level. Both treatments also 
significantly increased the proportion of claimants with multiple determinations and the average 
number of determinations per claimant. 

In both states, USA and IJSA+ also increased the rate of benefit denial. In D.C., the estimated 
increases, which are presented in Table VI.2, were 8 percentage points and 7 percentage points for 
USA and USA+, respectively. The estimated increases in Florida-3 percentage points for USA and 
2 percentage points for USA+— were smaller but still statistically significant. 

The impacts of USA and IJSA+ on determinations and denials tended to be smaller than the 
impacts of SJSA. For example, although the impacts of USA and USA+ on determinations (both 
equal to about 29 percentage points) were large in D.C., the impact of SJSA (37 percentage points) 
was still substantially larger. Similarly, the impact of SJSA on denials in D.C. was greater than the 
impact of USA or USA+. In Florida, the SJSA impacts were larger than the USA or USA+ impacts, 
but all of the impacts tended to be small, and the dilferences between the treatments in Florida were 
more modest than in D.C. 

The finding that USA and USA+ tended to generate smaller increases in determinations and 
denials than did SJSA is consistent with our expectations. SJSA required the average claimant to 
participate in more services and make more contacts with the demonstration than did the USA and 
USA+ treatments. Therefore, claimants in the SJSA group had a greater risk of jeopardizing their 
benefit eligibility by not complying with the treatment requirements. 
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TABLE VI.3 



ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH ASSISTANCE 
TREATMENTS ON NONSEPARATION, NONMONETARY 
DETERMINATIONS AND DENIALS, FLORIDA 







USA 


USA + 


Outcome Measure 


Control 

Group 

Outcome 


Treatment 

Group 

Outcome 


Estimated 

Impact 


Treatment 

Group 

Outcome 


Estimated 

Impact 


Determinations 












Percent with at Least One 


26.9 


29.5 


2.7** 

(1.2) 


29.6 


2.8*** 

(1.2) 


Percent with More Than One 


8.1 


10.1 


2.0*** 

(0.7) 


10.5 


2.4*** 

(0.7) 


Average Number per Claimant 


0.41 


0.47 


0.06*** 

(0.02) 


0.49 


0.09*** 

(0.02) 


Denials 












Percent With at Least One 


8.7 


11.7 


3.0*** 

(0.8) 


10.8 


2.0*** 

(0.8) 


Percent With more Than One 


2.2 


2.9 


0.7** 

(0.4) 


2.6 


0.5 

(0.4) 


Average Number per Claimant 


0.13 


0.17 


0.04*** 

(0.01) 


0.16 


0.03** 

(0.01) 


Percent of Determinations 
Leading to Denial 


31.3 


36.1 




32.2 




Sample Size 


3,014 


3,007 




2,989 





Source: State UI claims records. 

Note: The impact estimates are based on differences in outcomes between the control group and the 

IJSA and IJSA + groups. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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C. DETERMINATIONS AND DENIALS BY REASON 



In this section, we investigate the UI benefit determinations and denials by reason. Our 
investigation is intended to shed light on the way determinations and denials were used to enforce 
the demonstration participation requirements. 

1. Structured JSA 

We began our investigation of determinations and denials by examining the determinations by 
reason for the control group and SJSA group, which are shown in Table VIA. Since both states had 
a separate code for demonstration-related determinations and denials, we expected that the treatment- 
control differences would be fully explained by this code. Indeed, in D.C., much of the large overall 
difference in determinations between the control group and the SJSA group was attributable to the 
1,453 determinations related to JSA participation issues. These determinations represent more than 
half of all determinations conducted for the SJSA group, and they represent 91 percent of the overall 
difference between the groups in the number of determinations. 

But Table VIA also reveals one more notable difference between the D.C. groups. The number 
of determinations that occurred because claimants might have been employed (“not unemployed”) 
and therefore not eligible for UI was more than four times higher for the SJSA group than for the 
control group. This difference occurred even though “not unemployed” as a reason for the 
determination was not directly linked to the requirements of the demonstration. 

In Florida, there were many fewer determinations that were directly attributable to JSA than in 
D.C. Only 177 JSA-related benefit determinations were conducted in Florida. Furthermore, this 
difference accounts for only about half of the overall difference between the groups, which was equal 
to about 400 extra determinations for the SJSA group. The remaining difference is primarily a 
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Source: State UI claims records. 



result of the fact that the frequency of determinations made because a claimant was “able and 
available” was higher for the SJSA group than for the control group. 1 

Findings from both states on determinations by reason suggest that in both states, the local 
program operators used the extra information gathered as part of JSA enforcement to enforce other 
UI eligibility requirements. The demonstration therefore introduced two elements that affected 
determinations. First, it introduced new requirements that claimants needed to fulfill to avoid having 
their benefits denied. Second, it provided additional information that was used to check on other 
dimensions of UI eligibility. 

The effect of SJSA on determinations in D.C. also had important implications for benefit 
denials. As described above, denials were significantly more likely for the SJSA groups than for the 
control groups. Table VI.4 shows that the number of denials was twice as high for the SJSA group 
as for the control group. But only about a quarter of the difference is attributable directly to the JSA 
requirements. Most of the extra denials occurred because a claimant was “not unemployed.” The 
number of denials for this reason was four times higher for the SJSA group than for the control 
group. It therefore appears that in D.C. an important effect of the demonstration was to provide extra 
information on claimants who were found to be working and therefore ineligible for benefits. 

In Florida also, the difference in denials between the SJSA group and the control group is not 
primarily explained by benefits denied due to lack of JSA participation. The SJSA group had 529 
benefit denials compared with 384 for the control group. About one-third of the extra denials were 
tied to the JSA requirements, and about two-thirds were due to stricter enforcement of the “able and 
available” requirement for the SJSA group as compared with the control group. As in D.C., the 



‘“Able and available” issues relate to the requirement that a UI claimant be able and available 
to work in any week for which they claim benefits. 
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stricter enforcement of these eligibility requirements was probably made possible because of the 
additional information provided by the demonstration. 

2. Individualized JSA 

Again, the findings for individualized JSA parallel those for SJSA, but the differences are 
smaller in magnitude. Table VI.5 shows that many D.C. claimants assigned to the individualized 
treatments-848 in the USA group and 921 in the USA+ group-had benefit determinations that were 
directly related to the JSA participation requirements. But these figures are substantially lower than 
the 1,453 determinations for the SJSA group (Table VIA). That the USA and USA+ groups had 
fewer determinations is not surprising given that the SJSA treatments had more requirements for 
most claimants. We therefore expected more of the SJSA claimants to face benefit determinations 
due to nonparticipation in services. 

USA and USA+ claimants in D.C. were also more likely than the control group to have benefit 
determinations because they were deemed “not unemployed”. What was appears to be true for SJSA 
also appears to be tree for USA and IJSA+: that enforcement of the participation requirements 
complemented the enforcement of the traditional UI eligibility requirements. 

Some claimants in the Florida USA and IJSA+ groups had benefit determinations directly tied 
to the demonstration, although the number of those who did is substantially lower than in D.C. In 
Florida there were 84 determinations for the USA group and 102 for the USA+ group, which 
represents less than one-eighth of the corresponding demonstration-related determinations in D.C. 
shown in the same table. Not all of the treatment-control differences in determinations are due to 
the demonstration-specific determinations. The USA and IJSA+ groups had 278 and 230 “able and 
available” determinations, compared with the control group, which had only 168 such 
determinations. 
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The benefit denials for the USA and IJSA+ groups are organized by reason in Table VL6. In 
both states, the differences in benefit denials between the control group and the USA and USA+ 
groups were primarily attributable to issues not directly related to the demonstration. In D.C., most 
of the extra denials were due to stricter enforcement of the regular UI “not unemployed” 
requirement. In Florida, most of the extra denials were due to enforcement of the regular UI “able 
and available” requirement. These findings closely parallel our findings for the impacts of SJSA in 
both states, which also increased denials in D.C. due to the ‘"not unemployed” requirement and 
denials in Florida due to the “able and available” requirement. 

More benefit denials, whether or not directly related to the demonstration, account for some part of the 
reduction in UI receipt caused by the treatments. The increase in benefit denials for D.C. claimants who were 
employed is especially significant because this type of denial disqualifies the claimant for the entire claim rather 
than for a week of benefits, which is the case for most UI eligibility issues and was the case for JSA issues. Under 
a given set of assumptions, we calculated the UI savings generated by these denials to be 0.88 weeks per claimant 
in the SJSA group, which represents about 78 percent of the observed overall reduction of 1.13 weeks attributed 
reduction of 1.13 weeks attributed to SJSA. 2 Admittedly, this is a crude estimate and 



2 In D.C., 263 of 2,026 SJSA claimants, or 13.0 percent, were denied benefits because they were employed 
compared with only 62 of 2,012 control claimants, or 3.1 percent (these numbers do not match those in Table VI. 8 
because some claimants had multiple denials). So the net impact of SJSA was to increase the probability of this type 
of benefit denial by 9.9 percentage points. In our calculations, we assumed that the 20 1 additional SJSA claimants who 
were denied benefits because of demonstration requirements received 1 1.2 weeks of benefits, which is the average for 
all 263 SJSA claimants who were denied benefits because they were employed. We also assumed that in the absence 
of the demonstration, these claimants would not have been denied benefits and that they would have received the 
average UI amount for the control group, 20. 1 weeks. The UI savings generated by the demonstration-induced “not 
unemployed” denials is therefore (20.1 - 1 1.2) = 8.9 weeks per denied claimant. Multiplying the increased probability 
of a denial by the savings per denial yields: 8.9 *.099 = 0.88 weeks. Hence, the estimated reduction in UI weeks due 
just to denial for being employed is 0.88 weeks per claimant. 
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Source: State UI claims records. 



probably overestimates the true share of the impact due to benefit denials— similar calculations for the IJSA and 
USA+ groups generate UI savings due to denials that are larger than the estimated treatment impacts. Regardless, 
these calculations suggest that the increase in benefit denials probably played a substantial role in the UI reductions in D.C. 

Whether denials would play a similar role in an ongoing mandatory job search assistance program depends on 
how local staff enforce the participation requirements. If staff take the participation requirements seriously, we 
would expect that enforcement of the requirements would generate at least some benefit denials, which would 
contribute to the overall benefit reductions brought about by the job search assistance program. Alternatively, staff 
may choose not to enforce the requirements or may go to considerable lengths to avoid benefit denials, in which case 
we would not expect benefit denials to contribute substantially to program-induced UI reductions. 
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VII. IMPACTS ON EARNINGS AND EMPLOYMENT 



A key goal of the JSA demonstration was to measure the impacts of different mandatory 

0 ' 

treatments on claimants' earnings and employment experiences. The JSA demonstration was 
designed to quicken the pace of re-employment. If the demonstration was successful, members of 
the treatment groups should have had higher employment rates and earnings following their initial 
UI claim. In addition, if JSA enhanced the job search skills of claimants, they might have been able 
to locate higher-paying jobs, which would also translate into higher earnings. 

To test these hypotheses, we compared the earnings and employment experiences of claimants 
assigned to the JSA treatment groups with those of claimants assigned to the control group. Because 
the JSA demonstration randomly assigned claimants to treatment and control groups, we were able 
to use the differences between treatments and controls on various outcomes as valid estimates of 
program impacts. We estimated the quarterly impacts on earnings and employment using data from 
UI wage records. We analyzed these quarterly data starting with the first full quarter after the claim 
that triggered the claimant's participation in the demonstration. This approach gave us enough data 
to analyze employment and earnings for 10 quarters in the District of Columbia and 12 quarters in 
Florida. 

Our results suggest positive impacts of both structured and individualized treatments in D.C., 
and no impacts of these treatments in Florida. In D.C., the estimated impact of SJSA on earnings 
was approximately $200 per quarter, and the quarterly impact estimates are statistically significant 
at the 10 percent level for all quarters after the first. Following the first quarter, in which the 
estimated impact was small ($30) and insignificantly different from zero, the impacts were relatively 
stable over time. SJSA was also associated with a modest increase in the likelihood of being 
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employed in each quarter (about 2 to 3 percentage points), and the estimated impacts are statistically 
significant in about half of the quarters. Claimants in the individualized treatments (USA and 
IJSA+) had higher earnings and employment rates than controls in D.C., although the impacts were 
smaller than those found for SJSA. For instance, claimants in USA and USA+ in D.C. earned about 
$100 per quarter more than controls. The effects are statistically significant in about a third of the 
quarters. In Florida, JSA treatments did not have a statistically significant impact on earnings, 
although claimants in the USA and USA+ treatments were somewhat more likely than controls to 
be employed. 

In the remainder of this chapter, we discuss in more detail our estimates of the impacts of the 
JSA treatments on employment and earnings outcomes. Included in this discussion is an analysis 
of how the impacts vary across different subgroups of claimants. - 

A. INTERSTATE DIFFERENCES IN CONTROL GROUP OUTCOMES 

In this section, we describe the outcomes for members of the target population who were not 
required to participate in services, i.e. members of the control group. The target population in 
Florida fared better than the target population in D.C. Tables VII. 1 and VII.3 suggest that control 
group claimants in Florida had higher employment rates and earnings in each quarter following their 
initial UI claims than did their D.C. counterparts. Table VII.3 suggests that the higher quarterly 
earnings in Florida are attributable to higher employment rates in that state. It is difficult to know 
what effect this difference might have had on the estimated impacts of the JSA demonstration in the 
two states. It may have been more difficult for the JSA demonstration to register impacts in Florida 
if job opportunities were plentiful. In that case, the enhancement of job search skills and the increase 
in job search intensity brought about by the JSA demonstration may have been unnecessary because 
claimants would have found jobs rapidly even in the absence of JSA. 



O 

ERIC 



138 



177 



TABLE VH.1 



ESTIMATED IMPACTS OF STRUCTURED JSA ON QUARTERLY EARNINGS 

(Dollars per Claimant) 



Quarter 3 


District of Columbia 






Florida 




Control Group 
Mean 


SJSA Group 
Mean 


Estimated 

Impact 


Control Group 
Mean 


SJSA Group 
Mean 


Estimated 

Impact 


1 


1,258 


1,288 


30 


1,659 


1,712 


53 


2 


1,559 


1,731 


172** 


2,203 


2,199 


-4 


3 


1,767 


1,918 


152** 


2,592 


2,539 


-53 


4 


1,734 


2,015 


281*** 


2,673 


2,671 


-2 


5 


1,865 


2,145 


280*** 


2,881 


2,789 


-92 


6 


2,070 


2,312 


241** 


2,955 


2,890 


-66 


7 


2,161 


2,338 


177* 


3,034 


2,978 


-57 


8 


2,052 


2,315 


263** 


3,071 


2,974 


-98 


9 


2,102 


2,286 


185* 


3,165 


3,067 


-98 


10 


2,153 


2,378 


224** 


3,224 


3,201 


-23 


11 








2,914 


2,881 


-33 


12 








2,548 


2,427 


-121 


Total 






2,005*** 






-592 


Sample Siz 


2,00 

LLZ59 


2,02 

fl.772 




2,99 

( 2.551 


3,00 

( 2,561 





Source: State UI wage records. 

Note: The impact estimates are based on differences in outcomes between the control group and the 

treatment groups. 

a Full calendar quarters following initial UI claim. 

b Sample sizes are somewhat lower for the 10th quarter for D.C. and for the 12th quarter for Florida because data 
for claimants who entered the demonstration during the last quarter of the demonstration were not available. 

♦Statistically significant at the 90 percent confidence level in a one-tailed test. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

* * * Statistically significant at the 99 percent confidence level in a one-tailed test. 
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B. IMPACTS ON EARNINGS 



The JSA treatments were expected to affect earnings by quickening the pace at which claimants 
became re-employed and consequently increasing their total earnings. Whereas both the SJSA and 
IJSA treatments focused on providing job search skills, albeit in a different manner, the USA+ 
treatment was also supposed to emphasize job training when appropriate. By providing greater 
access to job training, the IJSA+ treatment was intended to enhance long-term earnings potential by 
making claimants more marketable, thereby increasing earnings. However, we found that the USA+ 
treatment did not increase the rate of participation in job training above that of claimants in the USA 
group, so we do not expect the employment and earnings impacts of these two groups to differ 
markedly. 

In this section, we present estimates of quarterly earnings for each of the JSA treatments and 
controls in D.C. and Florida. We estimated the impacts by comparing the quarterly earnings and 
total earnings of each of the JSA treatments to those of the control group. 

1. Structured JSA 

In D.C., SJSA had a positive impact on earnings. As shown in Table VII. 1, average earnings 
among SJSA claimants were higher than among the control group. Except for the first quarter, the 
differences are statistically significant. The impacts generally rose from $30 in the first quarter to 
$280 in the fourth and fifth quarters. The impacts in the sixth through tenth quarters were relatively 
stable, varying from $177 to $263. These results demonstrate that SJSA has a persistent effect on 
earnings in D.C. Overall, claimants in the SJSA treatment in D.C. earned $2,005 more than controls 

. - t ' 

during the 10-quarter observation period, a statistically significant difference. 

In Florida, the estimated impacts of SJSA are small and generally negative. Claimants in the 
SJSA earned $53 more than controls in the first quarter after filing for benefits and earned less than 
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controls in the remaining 11 quarters. However, none of the estimated impacts are statistically 
significant Overall, claimants in SJSA earned $592 less than claimants in the control group, but this 
difference is not statistically significant. SJSA appears to have had little impact on earnings in 
Florida. 

2. Individualized JSA 

The impacts of IJSA and USA+ on earnings in D.C. were somewhat smaller and less persistent 
than those found for SJSA (Table VII.2). The impacts are statistically significant in quarters four 
through six for IJSA, and for quarters two, three, and five for IJSA+. For both treatments, the 
impacts peaked in the fifth quarter ($191 for IJSA and $180 for USA+), and then declined. Overall, 
claimants in IJSA earned $1,171 more than controls, a statistically significant difference. Claimants 
in USA+ earned $789 more than controls, a difference that is not statistically significant. These 
results indicate that individualized job search assistance had positive impacts on quarterly earnings. 
However, these impacts are smaller than the impacts from SJSA, and they dissipate over time. 

Unlike the findings from D.C., there is no indication of any substantial impacts of USA or 
USA+ on earnings in Florida. Over the 12-quarter observation period, USA claimants earned $93 
less than controls, and IJSA+ claimants earned $25 more than controls. Neither of these differences 
is statistically significant, nor are any of the quarterly impact estimates. 



C. IMPACTS ON EMPLOYMENT RATES 

In this section, we discuss the impacts of JSA on the percentage of individuals who were 
employed in each quarter. If JSA treatments caused individuals to obtain jobs more quickly or to 
remain employed at higher rates, then we would expect quarterly employment rates to be higher for 
the treatment groups than for the control group. In each state, we determined which demonstration 
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TABLE VII.2 



ESTIMATED IMPACTS OF INDIVIDUALIZED JSA ON QUARTERLY EARNINGS 

(Dollars per Claimant) 



Quarter 8 


Control Group 
Mean 


USA 

Group Mean 


Estimated Impact 


IJSA+ 

Group Mean 


Estimated Impact 




H - '4 




in 


litl 


1 


1,258 


1,280 


22 


1,280 


22 


2 


1,559 


1,661 


102 


1,706 


147** 


3 


1,767 


1,877 


111 


1,942 


176** 


4 


1,734 


1,895 


161** 


1,817 


83 


5 


1,865 


2,056 


191** 


2,044 


* 

* 

o 

oo 


6 


2,070 


2,254 


183** 


2,176 


106 


7 


2,161 


2,257 


96 


2,138 


-23 


8 


2,052 


2,180 


129 


2,090 


38 


9 


2,102 


2,178 


76 


2,112 


10 


10 


2,153 


2,254 


100 


2,204 


50 


Total 






1,171* 




789 


Sample Size 


2,006 


2,018 




2,009 






(l,759) b 


(1,771 ) b 




(l,768) b 






mm mm 


i 


ft 

■ 


■KM ... 




1 


1,659 


1,611 


-48 


1,635 


-24 


2 


2,203 


2,197 


-6 


2,223 


20 


3 


2,592 


2,574 


- 18 


2,606 


14 


4 


2,673 


2,785 


112 


2,723 


50 


5 


2,881 


2,845 


-36 


2,868 


-12 


6 


2,955 


2,920 


-36 


2,961 


5 


7 


3,034 


3,029 . 


-5 


3,097 


63 


8 


3,071 


3,030 


-41 


3,051 


-20 


9 


3,165 


3,124 


-41 


3,116 


-49 


10 


3,224 


3,194 


-30 


3,180 


-44 


11 


2,914 


2,920 


6 


2,928 


14 


12 


2,548 


2,598 


50 


2,557 


9 



Total 






-93 




25 


Sample Size 


2,997 

(2,557) b 


2,993 

(2,602) b 




2,961 

(2,578) b 





Source ; State UI wage records. 

Note; The impact estimates are based on differences in outcomes between the control group and the treatment groups. 
a Full calendar quarters following initial UI claim. 



b Sample sizes are somewhat lower for the 10th quarter for DC and for the 12th quarter for Florida because data for claimants who 
entered the demonstration during the last quarter of the demonstration were not available. 

♦Statistically significant at the 90 percent confidence level in a one-tailed test. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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participants were employed in each quarter using UI wage records. We then compared the mean 
employment rates of the treatment and control groups to estimate the impacts of these JSA 
treatments on the employment rate. 

1. Structured JSA 

In D.C., the estimated impacts of SJSA on quarterly employment rates are consistently positive 
and range between 1.3 and 3.5 percentage points, as shown in Table VU.3. The impacts are 
statistically significant in quarters 1, 5,6 and 8 through 10, with the largest impacts occurring in the 
fifth and ninth quarters. The time pattern of the results is somewhat surprising because it was 
anticipated that SJSA would have had its largest impacts earlier as claimants influenced by the 
demonstration services moved back into employment. The results in Table VII.3 indicate that the 
effects of SJSA on employment were larger during the second year after claimants filed for benefits. 
As with earnings, the estimated quarterly impacts of SJSA on employment rates are insignificant for 
all 12 quarters in Florida. The estimated impacts range from -1.0 percentage points in quarter 12 to 
1.4 percentage points in quarter 1. 

2. Individualized JSA 

In general, for both DC and Florida, the estimated quarterly impacts of USA and IJSA+ on the 
employment rate are modest and statistically insignificant in most quarters. However, the impacts 
are largest and most consistently significant for IJSA in D.C. These impacts, shown in Table VH.4, 
are consistently positive and range from 1.0 percentage points in quarter 7 to 3.5 percentage points 
in quarter 6. The impacts in quarters 3 through 6 are larger and statistically significant. These 
results suggest that in D.C., USA took a couple of quarters to have an appreciable impact on 
employment, and that the impact diminished after the sixth quarter. In Florida, USA had a 




143 



182 



TABLE VII.3 



ESTIMATED IMPACTS OF STRUCTURED JSA ON 
PERCENT EMPLOYED 



Quarter 3 


District of Columbia 




Florida 




Control Group 
Mean 


SJSA 

Group Mean 


Estimated 

Impact 


Control 

Group 


SJSA 

Group Mean 


Estimated 

Impact 


1 


40.7 


42.9 


2.3* 


49.5 


50.9 


1.4 


2 


43.1 


44.6 


1.5 


57.7 


58.6 


0.8 


3 


45.2 


46.5 


1.3 


62.0 


62.5 


0.5 


4 


45.8 


47.7 


1.9 


63.8 


63.2 


-0.6 


5 


45.4 


48.9 


3.5** 


63.4 


62.8 


-0.6 


6 


45.5 


47.8 


2.3* 


62.8 


62.5 


-0.3 


7 


46.4 


47.6 


1.3 


63.4 


62.8 


-0.7 


8 


45.6 


48.0 


2.4* 


62.2 


62.6 


0.4 


9 


44.3 


47.4 


3.1** 


61.8 


62.8 


1.1 


10 


43.7 


46.5 


2.8* 


61.5 


62.2 


0.7 


11 








55.2 


55.4 


0.2 


12 








49.0 


48.1 


-1.0 


Sample Size 2,006 

- (l,759) b 


2,024 
fl. 772) b 




2,997 

(2,557) b 


3,009 

(2,567) b 





Source: State UI wage records. 

Note: The impact estimates are based on differences in outcomes between the control group and the 

treatment groups. 

a Full calendar quarters following initial UI claim. 

b Sample sizes are somewhat lower for the 10th quarter for D.C. and for the 12th quarter for Florida because data 
for claimants who entered the demonstration during the last quarter of the demonstration were not available. 

♦Statistically significant at the 90 percent confidence level in a one-tailed test. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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TABLE Vn.4 



ESTIMATED IMPACTS OF INDIVIDUALIZED JSA ON PERCENT EMPLOYED 



Quartet 


Control Group 
Mean 


IJSA 

Group Mean 


Estimated 

Impact 


IJSA+ 

Group Mean 


Estimated 

Impact 






WPm KDlSTRICT^Of 


BnnflEip 


mrmwsmm 




1 


40.7 


42.2 


1.5 


42.3 


1.6 


2 


43.1 


44.4 


1.2 


43.1 


-0.0 


3 


45.2 


48.1 


3.0" 


47.2 


2.0* 


4 


45.8 


48.5 


2.1** 


46.1 


0.3 


5 


45.4 


47.8 


2.5* 


46.3 


1.0 


6 


45.5 


48.9 


1.5** 


46.5 


1.1 


7 


46.4 


47.4 


1.0 


45.6 


-0.7 


8 


45.6 


47.2 


1.6 


44.8 


-0.8 


9 


44.3 


45.7 


1.4 


43.5 


-0.8 


10 


43.7 


44.9 


1.2 


43.9 


0.2 


Sample Size 


2,006 


2,018 




2,009 






(l,759) b 


(l,771) b 




(l,768) b 






i ' 9 






immi 


1 


49.5 


50.2 


0.7 


49,5 


0.0 


2 


57.7 


58.4 


0.7 


59.9 


2.2" 


3 


62.0 


63.5 


1.6 


64.0 


2.0^ 


4 


63.8 


65.4 


1.6 


63.9 


0.1 


5 


63.4 


64.0 


0.6 


63.1 


-0.3 


6 


62.8 


64.1 


1.3 


64.5 


1.6* 


7 


63.4 


64.0 


0.6 


64.0 


0.6 


8 


62.2 


63.8 


1.6 


63.6 


1.4 


9 


61.8 


63.8 


2.0^ 


62.2 


0.4 


10 


61.5 


63.2 


\.l* 


. 62.4 


0.9 


11 


55.2 


56.4 


1.2 


57.2 


2.0^ 


12 


49.0 


49.2 


0.1 


50:8 


1.8 


Sample Size 


2,997 
(2,557 ) b 


2,993 

(2,602) b 




2,961 

(2,578) b 





Source: State UI wage records. 

Note: The impact estimates are based on differences in outcomes between the control group and the treatment 

groups. 

“Full calendar quarters following initial UI claim. 

b Sample sizes are somewhat lower for the 10th quarter for D.C. and for the 12th quarter for Florida because data for 
claimants who entered the demonstration during the last quarter of the demonstration were not available. 

♦Statistically significant at the 90 percent confidence level in a one-tailed test. 

♦♦Statistically significant at the 95 percent confidence level in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent confidence level in a one-tailed test. 
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somewhat smaller and later impact on employment than in D.C. Only the impacts in quarters 9 and 
10 are statistically significant. However, unlike the impacts of SJSA in Florida, the impacts of USA 
are consistently positive, ranging from 0.1 percentage points to 2.0 percentage points. 

In D.C., the estimated impacts of USA+ are smaller than those observed for IJSA. Only the 
impact for quarter 3 is statistically significant, and nearly half of the estimated quarterly impacts are 
negative. In Florida, most of the estimated impacts of USA+ are positive, and four of the estimated 
impacts are statistically significant. However, no clear pattern in the timing of these effects is 
evident. The largest impacts occured both in early quarters (2 and 3) and in later quarters (1 1 and 
12 ). 

D. IMPACTS ON EARNINGS BY CLAIMANT SUBGROUP 

In sections B and C, we presented estimates of the average impacts of the JSA treatments on 
earnings. However, as for the impacts on UI outcomes, the impacts on labor market outcomes 
probably vary across different types of claimants. Therefore, we measured how these impacts vary 
according to basic demographic characteristics, pre-UI employment experience, and the predicted 
probability of exhausting UI benefits, as assigned in the profiling model. We found little compelling 
statistical evidence that the impacts of the treatments differ for different types of claimants. The 
impact estimates themselves often appear very different for different subgroups, but these differences 
were not measured precisely enough (due to small subgroup sample sizes) to identify many 
significant differences. The subgroup impacts for the initial benefit year are reported below. 

1. District of Columbia 

The results for the District of Columbia presented in Table VII.5 provide no compelling 
evidence that the impacts of the JSA treatments on earnings varied systematically across subgroups. 
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TABLE Vn.5 



SUBGROUP IMPACTS IN THE INITIAL BENEFIT YEAR, DISTRICT OF COLUMBIA: EARNINGS 

(Dollars per Claimant) 



Impacts 

Control 



Subgroups 


Mean 


SJSA 


IJSA 


IJSA+ 


Sample Size 


Average Impacts 3 




614* 


137 


197 


8,071 


Estimated Probability of Exhaustion 6 












1 st Quartile (lowest) 












2 nd Quartile 


6,973 


15 


78 


-241 


2,017 


3 rd Quartile 


7,181 


914 


605 


244 


2,019 


4 th Quartile (highest) 


5,426 


1,110* 


548 


975 


2,020 




5,750 


476 


296 


630 


2,015 


Gender 












Male 


5,612 


809 


116 


-186 


3,673 


Female 


6315 


450 


156 


516 


4,398 


Age 












Under 35 


5,699 


306 


368 


277 


3,398 


35 to 44 


6,135 


752 


147 


573 


2,448 


45 and older 


6394 


931 


-226 


-339 


2325 


Race/Ethnicity 












White 


7,574 


-694 


709 


-737 


744 


Black 


5,817 


823* 


174 


70 


5,916 


Hispanic 


6,441 


568 


-1,351 


228 


376 


Other 


5,716 


789 


64 


1,562* 


1,035 


Education 












No High School Diploma 


5,647 


124 


318 


-100 


1,140 


High School Diploma Only 


6,101 


635 


192 


66 


4,127 


College Degree 


6362 


168 


-91 


351 


2338 


Missing 


4,490 


3,174*# 


277 


1,122 


566 


Industry at Previous Job 












Wholesale and Retail Trade 


5,587 


-320 


-201 


880 


1,165 


Finance, Insurance, and Real Estate 


4316 


2,121* 


107 


-26 


655 


Services 


6,958 


608 


206 


189 


4321 


Public Administration 


2354 


-471 


-417 


886 


653 


Other 


5,963 


1317 


490 


-576 


1377 


Occupation at Previous Job 












Technical and Managerial 


5,446 


1335* 


464 


381 


1,664 


Clerical and Sales 


5,667 


512 


-92 


56 


4,769 


Other 


7,510 


281 


474 


419 


1,638 
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TABLE VU.5 (continued) 



Subgroups 


Control 

Mean 


SJSA 


Impacts 

USA 


IJSA+ 


Sample Size 


Tenure 


6,484 


719 


942* 


858 


2,673 


Less Than 1 Year 


6,020 


547 


119 


-149 


2322 


1 to 3 Years 


5,573 


149 


4% 


-23 


2,137 


3 to 10 Years 
10 Years or More 


5,504 


1,539* 


-668 


-328 


939 


Base Year Wages 


1 st Quartile (lowest) 


3,480 


116 


-637 


-84 


2,020 


2 nd Quartile 


5,498 


902 


-115 


430 


2,017 


3 rd Quartile 


6,864 


546 


107 


-62 


2,017 


4 th Quartile (highest) 


8,139 


891 


1,195*# 


1,364*# 


2,017 


Sample Size 


2,012 


2,026 


2,022 


2,011 





Source: State UI wage records. 

Note: The impact estimates are based on differences in outcomes between the control group and the treatment 

groups. 

a These impacts were evaluated at the mean over all eligibles for each of the subgroup variables listed in the table 
except for the probability of exhaustion (see footnote 2). 

b In computing the impacts for different categories of this variable, we did not control for the other subgroup variables 
listed in this table, many of which contribute to predicting exhaustion. 

♦Statistically different from zero at the 95 percent confidence level in a one-tailed test. 

#Statistically different from the average impact at the 95 percent confidence level in a two-tailed test. 
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The impacts do not seem to be related to the claimant’s exhaustion probability for any of the three 
treatments. Furthermore, one of the strongest subgroup results is perhaps the most puzzling: 
claimants with missing education data seemed to have benefitted the most from the JSA treatments, 
though the difference between this subgroup and the average claimant is only significant for SJSA. 
The most interesting subgroup impacts appear in the base year wages of claimants. Claimants in the 
top quartile of base year wages seem to have received much larger-than-average earnings gains from 
the individualized treatments. For SJSA, the earnings impact for the top quartile is very similar to 
the average impact ($891 versus $614, respectively). However, the impacts of the individualized 
treatments on earnings are much larger for the top quartile than for the average claimant. The 
impacts for the top quartile are $1,195 for IJSA and $1,364 for IJSA+, compared with average 
impacts of $ 1 37 for IJSA and $ 1 97 for USA+. 

In general, the subgroup analysis in D.C. provides very little evidence of interesting differences 
in the impacts of JSA treatments across different claimants. In fact, we found fewer subgroup impact 
estimates that differed significantly from the average impact than would be expected to occur by 
chance had the impact on earnings been the same for each claimant. 

2. Florida 

Like the results for D.C., the subgroup results for Florida (shown in Table VII.6) provide little 
compelling evidence that the impacts of JSA treatments vary systematically across claimants. For 
example, the impacts of the treatments do not seem to vary significantly according to the predicted 
probability of exhaustion. More generally, we found no more significant differences between 
different subgroups than would be expected to occur by chance if the impacts were the same for all 
claimants. Therefore, the few apparently interesting subgroup results described below should be 
interpreted with cautioa The individualized treatments seemed to raise earnings for women but 
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TABLE VII.6 



SUBGROUP IMPACTS IN THE INITIAL BENEFIT YEAR, FLORIDA: EARNINGS 

(Dollars per Claimant) 



Impacts 



Subgroups 


Control 

Mean 


SJSA 


USA 


IJSA+ 


Sample 

Size 


Average Impacts 3 




49 


194 


10 


12,042 


Estimated Probability of Exhaustion b 
l a Quartile (lowest) 

2 nd Quartile 


8,950 


755 


-191 


605 


3,005 


3 rd Quartile 


9,162 


-395 


-388 


-394 


3,016 


4 th Quartile (highest) 


9,607 


-256 


-126 


-144 


3,012 




. 8,825 


-158 


804 


139 


3,009 


Gender 


Male 


10,133 


20 


-159 


-662# 


6,504 


Female 


7,993 


84 


609 


799*# 


5,538 


Age 


Under 35 


8,994 


599 


65 


701 


3,425 


35 to 44 


9,840 


-34 


465 


-70 


3,424 


45 and older 


8,796 


-258 


101 


. -391 


5,193 


Race/Ethnicity 


White 


9,524 


-171 


-58 


-76 


7,401 


Black 


8,914 


262 


-16 


-506 


1,880 


Hispanic 


8,274 


655 


1,030* 


726 


2,614 


Other 


8,788 


-2360 


728 


-1,801 


147 


Education 

No High School Diploma 


7,957 


22 


429 


120 


3,122 


High School Diploma Only 


8,907 


263 


-52 


181 


6,692 


College Degree 


11,543 


-554 


605 


-656 


2328 


Industry at Previous Job 


Wholesale and Retail Trade 


8,589 


1,133* 


1,027 


384 


2,012 


Finance, Insurance, and Real Estate 


9,457 


732 


1,155 


995 


1381 


Services 


9,175 


-51 


-416 


-184 


3300 


Manufacturing 


10,697 


-1386* 


-55 


-1,492*# 


1,551 


Other 


8,709 


-115 


54 


247 


3,998 


Occupation at Previous Job 


Technical and Managerial 


10,248 


38 


800 


-54 


2,460 


Clerical and Sales 


9,076 


277 


-140 


-342 


5,365 


Other 


8,601 


-234 


267 


497 


4317 
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TABLE VII.6 (< continued ) 



Subgroups 


Control 

Mean 


SJSA 


Impacts 

1JSA 


IJSA+ 


Sample 

Size 


Tenure 












Less Than 1 Year 


11,295 


452 


-608# 


-273 


4,372 


1 to 3 Years 


8,633 


614 


1,041* 


-330 


3,005 


3 to 10 Years 


7,653 


79 


217 


88 


3,464 


10 Years or More 


6,973 


366 


916 


1,663*# 


1,201 


Base Year Wages 












1 st Quartile (lowest) 


5,202 


572 


387 


96 


3,019 


2 nd Quartile 


7,463 


136 


-8 


110 


3,008 


3 rd Quartile 


9,397 


194 


655 


564 


3,008 


4 th Quartile (highest) 


14,533 


- -706 


-257 


-730 


3,007 


Sample Size 


3,014 


3,032 


3,007 


2,989 





Source: State UI wage records. 

Note: The impact estimates are based on differences in outcomes between the control group and the treatment 

groups. 

a These impacts were evaluated at the mean over all eligibles for each of the subgroup variables listed in the table except 
for the probability of exhaustion (see footnote 2). 

b In computing the impacts for different categories of this variable, we did not control for the other subgroup variables 
listed in this table, many of which contribute to predicting exhaustion. 

* Statistically different from zero at the 95 percent confidence level in a one-tailed test. 

#Statistically different from the average impact at the 95 percent confidence level in a two-tailed test. 
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lower earnings for men. The impacts of IJSA+ are +$799 for women and -$662 for men, and the 
difference is statistically significant. We found a similar, though smaller, difference in impacts by 
gender for USA. 

The individualized treatments also seemed to produce larger earnings impacts for claimants with 
longer pre-UI job tenure than for claimants with shorter job tenure. The impact of USA on earnings 
is negative only for claimants with less than one year of pre-UI job tenure (-$608), and this estimate 
is significantly different from our estimate of the average impact across all job tenure categories 
(+$194). The inpact of USA+ is only positive and significantly different from zero for claimants 
with 10 or more years of job tenure (+$1,663), and this estimate is significantly different from our 
estimate of the average impact (+$10). . . 
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VIII. IMPACTS ON POST-UI JOB CHARACTERISTICS 



Another way to judge the impact of JSA is to look at the characteristics of jobs obtained by JSA 
claimants. We are specifically interested in those job characteristics that lend some insight into the 
quality of post-UI employment. Although a number of factors can be considered indicators of job 
quality, we consider wages and fringe benefits to be good indicators of job quality as well as being 
easy to measure. Consequently, we gauge impacts based on these outcomes. 

The anticipated impact of JSA on the quality of post-UI employment is ambiguous. By 
encouraging rapid re-employment, JSA could cause some claimants to take “any” job and might 
therefore result in lower quality post-UI employment. Alternatively, by improving the job search 
skills of claimants, JSA could enable them to find higher quality jobs. One evaluation of a JSA-type 
program in New Jersey (Corson et al. 1989) considered the impact of JSA on job quality by 
analyzing hourly wages in post-UI jobs. In this chapter, we paint a fuller picture of the impacts of 
JSA on job quality by considering the impacts of JSA on claimants’ health insurance and pension 
benefits in addition to hourly wages. We use health insurance and pension benefits to measure job 
quality because these are probably the two most important fringe benefits in terms of contributing 
to an employee’s well-being. 

In addition to the relationship between JSA and job quality, we also consider whether JSA 
increases the likelihood of claimants switching to a new occupation. Although not strictly a measure 
of job quality, changing occupations is an important possible outcome, especially because the type 
of worker targeted by the JSA demonstration may have skills that are no longer in demand and may 
therefore need to switch occupations. Like employment in general, the anticipated effect of JSA on 
the occupations of claimants is also ambiguous. On one hand, we might expect claimants who were 
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exposed to the JSA treatment to be more aware of opportunities in other occupations and thus more 
likely to switch occupations. Alternatively, the job search skills claimants acquire may enable them 
to find employment in their old occupation, even if there are now fewer opportunities in that 
occupation. 

Our methods for assessing the impacts of JSA on post-UI job characteristics differ from the 
methods we used to obtain estimates of program impacts. For those other outcomes, the random 
assignment of claimants to treatment and control groups allowed us to make unbiased estimates of 
the impact of JSA. But here, the outcomes of interest are the characteristics of the claimant’s post- 
UI job, and this limits our sample to those who obtained a job, potentially invalidating the treatment 
and control groups for estimating program impacts. To control for observable differences that* may 
exist between the employed treatment group and control group members, we used multivariate 
regression techniques. To the extent that unobservable differences still exist between the treatments 
and the controls, our results may remain somewhat biased. 

To examine the impacts of the JSA demonstration on job characteristics, we focused on the job 

claimants held one year after their initial UI claim. Information about this job was drawn from the 
follow-up survey. Survey respondents who did not hold a job one year after their initial UI claim 
were not included. Since the analysis is based only on survey respondents who held a job one year 
after their ini tial claim, the sample sizes are relatively small, and the estimates are imprecise. ' 
Keeping this in mind, we still note substantial impacts even when they fail to achieve conventional 
levels of statistical significance. 

Another issue associated with using the survey data is the potential for nonresponse bias. In 
Appendix A, we evaluate the potential effects of survey nonresponse and find some evidence that 
treatment impacts on outcomes associated with employment and earnings may be somewhat 
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overstated when based on the survey sample. Since some data, such as information on job 
characteristics, are available only from the survey, we still want to use the survey sample for some 
pieces of analysis. However, in interpreting the findings, we are cognizant of the fact that 
nonresponse bias may prompt us to overstate the impacts. 

Our estimates provide little evidence that the JSA treatments encouraged clients to take lower- 
quality jobs than they would have without the treatments. Typically, the estimated treatment impacts 
on hourly wages and job benefits were positive and statistically insignificant. Some scattered 
impacts were positive and significant. For example, SJSA in D.C. generated an estimated increase 
in hourly wages of $0.99. The structured and individualized treatments each generated a significant 
increase in the rate at which claimants received health insurance benefits or pension benefits in at 
least one state. Based on these findings, we conclude that the treatments, if anything, led claimants 
to take higher-quality jobs rather than lower-quality jobs. 

Our findings also suggest that the JSA had relatively little impact on occupational choice. When 
claimants had difficulty obtaining employment in the same occupation they had before they were laid 
off, JSA does not appear to have lessened this hardship. 

Viewed together, these findings suggest that claimants in the treatment groups either started 
their job search earlier or searched more intensively, but that they did not lower their reservation 
wage or any other standard for an acceptable job offer. The positive impacts on hourly wages for 
SJSA in D.C. are probably at least partially responsible for the persistent earnings impacts found for 
SJSA, as described in the previous chapter. These findings suggest that claimants in SJSA in D.C. 
were induced to find relatively permanent, higher-paying jobs. 
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A. IMPACTS ON HOURLY WAGES 



The primary indicator of job quality is the hourly wage. The regression-based estimate 
presented in Table VIII. 1 implies that SJSA increased the post-UI hourly wages of claimants in D.C. 
by $0.99. This estimated impact is both large, representing nearly 10 percent of the control group 
mean ($10.61), and statistically significant at the 99 percent confidence level. In contrast, the 
employed SJSA claimants in Florida experienced an impact on the hourly wage of only $0.02, which 
is not statistically significant. These results are consistent with the results we presented in Chapter 
VII that showed significant impacts due to SJSA on quarterly earnings in D.C. but little evidence of 
impacts in Florida. These results also suggest that the impacts on earnings in D.C. were due to both 
impacts on wages (presented here) and impacts on employment (presented in Chapter VII). 

For the individualized treatments, we present estimated impacts based on combining the USA 
and USA+ groups. This method of presentation is based explicitly on the design of the survey. 
Since there was little distinction between USA and USA+ in the field, the survey was designed so 
that the USA and USA+ groups were interviewed at half the rate of the SJSA and control groups. 
The intent was to combine the USA and USA+ respondents into a single group comparable in size 
to the SJSA or control group respondents. 

Our estimates provide no clear evidence that the individualized treatments had any effect on 
wages. Other things being equal, hourly wages were generally higher for the combined 
individualized treatment groups than for the control group, by $0.07 in D:C. and by $0.57 in Florida, 
but neither difference is statistically significant 
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TABLE YIU.1 



ESTIMATED IMPACTS OF TREATMENTS ON HOURLY 
WAGES IN POST-UI JOB 
(Standard Errors in Parentheses) 





District of Columbia 


Florida 


SJSA Impact 


$099*** 


$0.02 




(0.37) 


(0.38) 


Combined USA and USA+ Impact 


$0.07 


$0.57 




(0.37) 


(0.38) 


Control Group Mean 


$10.61 


$10.19 


Sample Size 


901 


1,260 



Source: JSA demonstration follow-up survey. 

* * * Statistically significant at the 99 percent confidence level in a two-tailed test. ; 
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B. IMPACTS ON HEALTH INSURANCE BENEFITS 

Recent policy debates on the financing of health care attest to the importance of this benefit. 
For many employees, it gauges of the quality of post-UI employment. In this section, we examine 
the impact of JSA on health insurance benefits. 

Table VIII.2 shows that SJSA had a sizable impact on the likelihood of D.C. claimants obtaining 
a job with health insurance benefits. The SJSA claimants had an 11.3 percentage point greater 
probability of obtaining a job that provided health insurance than the control group; this impact is 
statistically significant at the 99 percent confidence level. In Florida, however, there is no evidence 
that SJSA increased the likelihood that claimants would obtain a job with health insurance benefits. 
On the contrary, the SJSA claimants had a 1.3 percentage point lower probability of obtaining a 
position with health benefits, but the difference is statistically insignificant 

The SJSA impacts on health insurance benefits parallel those for hourly wages. Although the 
small sample size makes our estimates less than definitive, the evidence leans toward suggesting 
that in D.C., SJSA claimants earned higher hourly wages and were more likely than the control group 
claimants to receive health benefits. In Florida, no positive impacts were found for either outcome. 
These results suggest that the SJSA treatment in D.C. was effective in getting claimants into better 
jobs, while this was not the case in Florida. This may be an indication that D.C. has more jobs with 
high wages and health benefits than Florida, and that these jobs are available to UI claimants who 
are encouraged to search intensely for re-employment. 

The claimants assigned to the individualized treatments tended to find jobs with health insurance 
at a higher rate than did the control group, as reflected in the positive impact estimates shown in 
Table Vin.2. The estimates tend to be fairly large— about 10 percentage points in D.C. and 5 
percentage points in Florida. However, only the D.C. estimate is statistically significant. The 



TABLE Vni.2 



ESTIMATED IMPACTS OF TREATMENTS ON PROBABILITY OF 
RECEIVING HEALTH INSURANCE ON POST-UI JOB 
(Percentage Points) 





District of Columbia 


Florida 


SJSA Impact 


11.3*** 


-1.3 


Combined USA and USA+ Impact 


9.6** 


4.6 


Control Group Mean (Percent) 


40.1 


38.0 


Sample Size 


945 


L316 



Source: JSA demonstration follow-up survey. 

Note: Impacts shown are the marginal effects of the treatments calculated based on logit 

regression estimates. Statistical significance results are based on hypothesis tests of the 
logit coefficients. 

“Statistically significant at the 95 percent confidence level in a two-tailed test. 

'“Statistically significant at the 99 percent confidence level in a two-tailed test. 
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substantial but insignificant estimate for Florida reflects the limitations created by the relatively 
small survey samples and the resulting imprecision of our estimates. 

C. IMPACTS ON PENSION BENEFITS 

Pension benefits represent a significant source of income for many retired Americans. Pension 
benefits can determine whether workers spend their retirement years comfortably or in poverty, or 
they can determine whether workers are able to retire at all. Given the importance of this benefit, 
it can serve as an important indicator of job quality. In this section, we examine the impact of JSA 
on pension benefits. 

Table VHI.3 shows the SJSA treatment group in D.C. had a probability of obtaining a job with 
pension benefits that was 4.9 percentage points higher than the control group’s probability of 
obtaining such a job; the corresponding figure in Florida is 4.0 percentage points. These impacts, 
however, are not statistically significant. Nevertheless, the magnitude of the point estimates is not 
trivial, as the estimates represent up to a 20 percent increase over the control group mean. Given the 
small sample sizes for our survey sample, our estimates are relatively imprecise, and it is not 
surprising that even sizable estimates therefore become statistically insignificant. 

The results for pension benefits presented in Table VHI.3 are consistent with the SJSA impacts 
we observed in D.C. for hourly wages and health insurance benefits, where the impacts observed for 
both were substantial. Thus, the substantial, albeit insignificant, impacts for pension benefits in D.C. 
are consistent with the overall pattern. In Florida, the impact of SJSA on pension benefits runs 
somewhat counter to what we observed on other measures of job quality. On both hourly wages and 
health insurance benefits, the SJSA had a zero or even a negative, albeit insignificant, impact. In the 
case of pension benefits in Florida, the impact was positive, although also insignificant 
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TABLE VIU.3 



ESTIMATED IMPACTS OF TREATMENTS ON PROBABILITY OF 
RECEIVING PENSION BENEFITS ON POST-UI JOB 
(Percentage Points) 





District of Columbia 


Florida 


SJSA Impact 


4.9 


4.0 


Combined IJSA and USA+ Impact 


8.2** 


8.5*** 


Control Group Mean (Percent) 


23.2 


24.7 


Sample Size 


947 


1,316 



Source: JSA demonstration follow-up survey. 



Note: Impacts shown are the marginal effects of the treatments calculated based on logit 

regression estimates. Statistical significance results are based on hypothesis tests of the 
logit coefficients. 

* * Statistically significant at the 95 percent confidence level in a two-tailed test. 

***Statistically significant at«the 99 percent confidence level in a two-tailed test. 
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The individualized treatments appear to have increased the likelihood of receiving pension 
benefits on the new job. Both of the estimated impacts of the combined individualized treatments 
shown in Table VHI.3 are positive and statistically significant at least at the 95 percent confidence 
level. These estimates suggest that the treatments increased pension benefit receipt by about 8 
percentage points in both states. 

D. IMPACTS ON OCCUPATIONAL RETENTION 

Of particular interest to policymakers is whether the services provided in the JSA demonstration 
led workers to switch occupations. Given that many of the claimants were dislocated workers, 
choosing another occupation might be a fruitful strategy for obtaining employment. On the other 
hand, the JSA demonstration might have enabled claimants to locate hard-to-fmd jobs in their old 
occupations. In this section, we address these questions by analyzing the impacts of the JSA 
demonstration on occupations. 

Before estimating the impacts of JSA on occupational retention, we examine the post-UI 
occupational distributions of the JSA claimants by treatment and control group. The occupational 
distributions, which are shown in Table Vni.4, tend to be broadly similar across the groups, with the 
magnitude of any differences being fairly small. These statistics suggest that none of the treatments 
had much of an impact on occupations. In D.C., the most popular post-UI occupations across all 
groups were professional, clerical, and service jobs. Clerical jobs were also popular in Florida, but 
the claimants in other jobs, excluding laborers, were spread fairly evenly across the remaining 
occupations. 

We addressed the issue of occupational retention by estimating the impacts of the JSA 
treatments on the likelihood that claimants found new jobs in the same occupations as their old jobs. 



TABLE VIII.4 



POST-UI OCCUPATIONAL DISTRIBUTIONS BY TREATMENT 
AND CONTROL GROUP 
(Percent) 



Occupation 




Control Group 


SJSA Group 


Combined IJSA and 
IJSA+ Group 




- J8if 


S3'’ ' WI& * 

Km& :--^tv-Disrwi8T:or Columbia Mbs 




Professional 




14.4 


16.9 


12.8 


Technical & Sales 




1.0 


2.1 


3.4 


Clerical 




44.1 


41.2 


40.2 


Service 




23.3 


23.5 


28.6 


Crafts 




7.9 


9.9 


6.8 


Operator 




1.5 


1.2 


1.7 


Laborer 




6.9 


4.1 


6.4 


Sample Size 




202 


243 


234 




■mm 




’-SlI/SiE lllfi 


Professional 




13.4 


14.6 


13.0 


Technical & Sales 




18.9 


20.0 


16.5 


Clerical 




26.0 


25.6 


26.5 


Service 




14.8 


13.4 


14.6 


Crafts 




8.5 


10.1 


12.3 


Operator 




13.4 


12.6 


12.5 


Laborer 




3.2 


3.5 


3.2 


Sample Size 




411 


454 


431 



Source: JSA demonstration follow-up survey. 
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Table VDI.5 shows that in D.C., there was not much of an impact on occupational retention. 
SJSA decreased the probability of claimants remaining in the same occupation by 2.5 percentage 
points, but this impact was not statistically significant. In Florida, the impact was also negative, with 
SJSA decreasing the probability of claimants obtaining a job in the same occupation by 1.5 
percentage points, but again the impact was statistically insignificant. This absence of substantial 
change in the occupational distribution in conjunction with the absence of an impact on the retention 
rate leads us to conclude the SJSA treatment did not substantially affect the occupational focus of 
job search activities. 

Table VHI.5 presents the impacts of the combined individualized treatments on occupational 
retention rates. According to these results, the combined treatments did not have a substantial impact 
on the retention rate in either D.C. or Florida. 



TABLE VIII.5 

ESTIMATED IMPACTS OF TREATMENTS ON PROBABILITY 
OF OCCUPATIONAL RETENTION 
(Percentage Points) 



Treatment 


District of Columbia 


Florida 


SJSA Impact 


-2.5 


-1.5 


Combined USA and IJSA+ Impact 


1 2 


1.4 


Control Group Mean (Percent) 


78.4 


62.9 


Sample Size 


730 


1,294 



Source: JSA demonstration follow-up survey. 

Note: Impacts shown are the marginal effects of the treatments calculated based on logit regression 

estimates. Statistical significance results are based on hypothesis tests of the logit coefficients. 
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TABLE VIII. 5 



ESTIMATED IMPACTS ON PROBABILITY OF OCCUPATION RETENTION 

(Percentage Points) 



Treatment 


District of Columbia 


Florida 


SJSA Impact 


-2.5 


-1.5 


IJSA Impact 


-0.6 


-1.2 


USA+ Impact 


3.4 


4.2 


Combined USA and USA+ Impact 


1.2 


1.4 


Control Group Mean (Percent) 


78.4 


62.9 


Sample Size 


730 


1,294 



Source: JSA demonstration follow-up survey. 

Note: Impacts shown are the marginal effects of the treatments calculated based on logit 

. regression estimates. Statistical significance results are based on hypothesis tests of 
the logit coefficients. 
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IX. IMPACTS ON JOB SEARCH ACTIVITIES 



Improving the skill and aggressiveness with which claimants search for jobs is a key objective 
of the JSA demonstration. Indeed, the improvement of job search efforts, which in turn leads to 
more rapid re-employment and a better “match” between job seekers’ skills and the jobs they 
ultimately land, can be viewed as the direct objective of the JSA demonstration. To the extent the 
JSA demonstration works the way it is intended, changes in UI receipt, employment, and earnings 
are the ultimate results of the demonstration’s intensified job search activities. 

In previous chapters, we have noted that the JSA treatments decreased UI receipt and increased 
employment-related outcomes. This finding provides some indication that the treatments may have 
increased the intensity of claimants’ job search efforts. But since findings presented in earlier 
chapters also suggest that the effects of JSA on UI receipt were at least partly due to the increased 
enforcement of UI eligibility rules, it is possible that the impacts on UI receipt are not attributable 
to an increase in search intensity. To investigate this issue in greater detail, we estimated the impacts 
of the JSA treatments on job search behavior using measures of job search activities drawn from the 
follow-up survey. 

The estimates presented in this chapter provide some evidence that the JSA demonstration 
encouraged more aggressive job searches. More specifically, in both D.C. and Florida, the structured 
treatment (SJSA) led claimants to contact more prospective employers per week. In D.C., the 
individualized treatments (USA and USA+) increased both the likelihood that claimants searched 
for employment following their job loss and the number of employer contacts per week. In Florida, 
the individualized treatments increased both contacts and hours of search per week. The other 
estimated impacts on job search were less consistent and less precise. 



Another potential impact of the demonstration was to increase claimants’ use of the state Job 
Service (JS) to assist in their job search. We find that the treatments consistently increased the 
likelihood that claimants used their state JS. This is not surprising given that the treatments were 
specifically designed to encourage claimants to use the JS. The treatments also tended to increase 
the referrals that claimants received from the JS. However, there was no evidence of impacts on the 
likelihood of obtaining job offers through JS referrals. Thus, it appears that the JSA demonstration 
was successful in getting claimants to use the state JS, but less successful in translating these services 
into employment opportunities. 

A. IMPACTS ON JOB SEARCH EFFORT 

We examined job search effort using three measures that were created from claimant responses 
to the follow-up survey. The three measures are (1) whether claimants searched for employment 
after their job loss, (2) hours per week spent searching for work, and (3) number of employers 
contacted per week. We estimated impacts on all of these measures by calculating the difference 
between mean outcomes for the treatment and control groups. 1 When we investigated survey-based 
job characteristic measures in the previous chapters, we combined the individualized treatments to 
•create reasonable sample sizes. In this chapter, our sample sizes are larger than in the previous 
chapter, since we have job search measures for all survey respondents rather than just employed 
respondents. Given these larger sample sizes, we present estimated impacts separately for USA and 
USA+, as well as estimated impacts for USA and USA+ combined. 



1 Since these estimates are based on the survey, they are subject to potential survey nonresponse bias. In an effort 
to control for nonresponse bias as much as possible, we generated alternative estimates based on regressions that control 
fora variety of individual factors as determinants ofthe job search measures. We found that these regression estimates 
were similar to the treatment-control differences and therefore chose not to present them. 
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1. Structured Job Search Assistance 



SJSA may have had some positive effect on the job search effort of claimants, but the evidence 
is not consistently strong. For the three measures that we report on in Table IX. 1, the only sizable 
increases in job search effort occurred for the number of employer contacts per week. SJSA 
increased contacts in D.C. and Florida by an estimated 1.6 and 1.4 contacts per week, respectively. 
These statistically significant estimates represent roughly a 10 percent increase in contacts for the 
average claimant. The estimated impacts on the other measures provide no clear evidence of 
increased search effort: SJSA had no statistically significant effect on the likelihood that claimants 
searched for work or the hours per week they spent searching. 

Given that SJSA had only modest effects on employment and earnings, as discussed in Chapter 
VII, we are not surprised to find that it had a modest impact on search effort. SJSA could have also 
potentially generated more effective job searches without increasing effort. For example, claimants 
who attended the job search workshop may have learned how to target their efforts more efficiently 
or communicate more effectively with potential employers. Evaluating the effectiveness of a 
claimant’s job search, holding effort constant, would require a more detailed investigation and is 
beyond the scope of this study. However, the modest impacts of SJSA on employment and earnings 
(described in Chapter VII) suggest that any impacts on search effectiveness were limited. 

2. Individualized Job Search Assistance Treatments 

Among the individualized job search assistance treatments, the strongest evidence of positive 
impacts on effort occurred for USA+ in Florida. Estimates presented in Table IX.2 show that USA+ 
increased the likelihood that claimants searched for work by 3 percentage points, increased average 
hours of search by 1.7 hours per week, and increased contacts by 2.1 contacts per week. All of these 
estimates are statistically significant at least at the 95 percent level of confidence. The estimated 
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TABLE IX. 1 



ESTIMATED IMPACTS OF STRUCTURED JOB SEARCH ASSISTANCE 
ON JOB SEARCH EFFORT 







District of Columbia 




Florida 




Job Search Measure 


Control 

Group 

Mean 


SJSA 

Group 

Mean 


Estimated 

Impact 


Control 

Group 

Mean 


SJSA 

Group 

Mean 


Estimated 

Impact 


Percent Who Searched for 


89.5 


89.9 


0.4 


95.5 


94.9 


-0.6 


Employment 


(1.5) 


(1.5) 


(2.1) 


(0.8) 


(0.8) 


(1.2) 


Hours Per Week Searching 


14.3 


14.5 


0.2 


14.8 


15.2 


0.4 


for Employment 


(0.6) 


(0.6) 


(0.8) 


(0.5) 


(0.4) 


(0.6) 


Contacts Per Week 


13.1 


14.7 


1.6* 


13.2 


14.5 


1.4" 




(0.7) 


(0,9) 


(1.2) 


(0-5) 


(0.6) 


(0.7) 


Sample Size 


419 


425 




640 


703 





Source: JSA demonstration follow-up survey. 



Note: The impact estimates are based on differences in outcomes between the control group and the treatment 

groups. 

♦Statistically significant at the 90 percent level of confidence in a one-tailed test. 

♦♦Statistically significant at the 95 percent level of confidence in a one-tailed test. 
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ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH ASSISTANCE 
ON JOB SEARCH EFFORT 
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impacts on search hours and employer contacts per week represent increases of more than 10 percent 
above the control group means. 

For the other treatments, USA in Florida and USA and USA+ in D.C., the evidence is somewhat 
less clear that search intensity increased in response to the services. In D.C., USA increased the 
claimants who reported they searched for work by 4.8 percentage points, and this impact is 
significant at the 95 percent confidence level. The estimates presented in Table IX.2 also imply that 
in D.C. the USA and USA+ treatments increased average employer contacts per week by 1.9 and 3.0, 
respectively. These impacts are substantial, representing a 15 to 20 percent increase in contacts, and 
statistically significant at the 90 percent confidence level. Similarly, the estimates for USA in 
Florida suggest that the treatment generated 1.5 extra employer contacts per week, on average. 

The lack of consistently significant impacts in Table IX.2 is due partly to the small size of these 

samples. In the case of the USA and USA+ treatments, samples are especially small because of the 

* 

survey’s design. Our goal in the survey was to interview the USA and USA+ groups at half the rate 
of the SJSA and control groups so the USA and IJSA+ respondents could be combined for the data 
analysis into a single group comparable in size to the SJSA or control group respondents. This is 
reflected in the sample sizes shown in Table IX.2, which show that in Florida, for example, we have 
survey responses from 640 control group members, compared with 355 USA group members and 
321 USA+ group members. 

Given the similarity in the services offered to and received by USA and USA+ claimants, we 
combined these groups in our analysis to increase our statistical power in evaluating the impacts of 
the USA treatments. The final column in Table IX.2 shows the estimated impacts based on these 
combined groups. The combined USA treatments generated 2.4 extra employer contacts per week 
in D.C. and 1.8 extra contacts in Florida. Both of these estimates are statistically significant at the 
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95 percent level. The combined IJSA treatments also generated a statistically significant increase 
in the percentage of D.C. claimants who searched for employment and in the average time Florida 
claimants spent searching per week. The remaining estimated impacts are all positive but not 
statistically significant. 

B. IMPACTS ON CONTACTS WITH STATE EMPLOYMENT SERVICE 

One of the JSA demonstration’s main strategies to promote more effective job searches was to 
increase use of the state JS. In this section, we explore the impact of the JSA treatments on 
claimants’ experiences with the state JS. Our analysis focuses on three measures: (1) whether 
claimants reported contact with the JS, (2) whether claimants received job referrals through the JS, 
and (3) whether claimants received job offers based on referrals from the JS. 

1. Structured Job Search Assistance 

SJSA increased reported contact with the JS in both states. This is not surprising because SJSA 
required that claimants participate in services offered through the JS. In D.C., the rate at which 
claimants reported contact with the JS increased by 10.0 percentage points in response to SJSA, 
while the corresponding increase in Florida was about 6.3 percentage points. Both of these estimates 
are statistically significant at the 99 percent confidence level. 

One interesting point, based on Table IX.3, is that although the treatment groups had higher rates 
of JS contact, a high proportion of control group members also reported JS contact. In D.C., 44.9 
percent of the control group reported contact with the JS, while in Florida, 70.9 percent of the control 
group reported similar contact. These high rates among control groups in both states are not 
surprising given that both states require most new claimants to at least register with the JS. 
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TABLE IX.3 



ESTIMATED IMPACTS OF STRUCTURED JOB SEARCH ASSISTANCE 
ON USE OF STATE JOB SERVICE 







District of Columbia 




Florida 




Measure of Job Service Use 


Control 

Group 

Mean 


SJSA Group 
Mean 


Estimated 

Impact 


Control 

Group 

Mean 


SJSA Group 
Mean 


Estimated 

Impact 


Percent Who Contacted 


44.9 


54.8 


10.0*** 


70.9 


77.2 


6.3*** 


State Job Service 


(2.4) 


(2.4) 


(3.4) 


(1.8) 


(1.6) 


(2.4) 


Percent Who Received Job 


15.5 


24.2 


8.7*** 


28.9 


32.3 


3.4* 


Referrals from State Job 
Service 


(1.8) 


(2.1) 


(2.7) 


(1.8) 


(1.8) 


(2.5) 


Percent Who Received Job 


2.9 


2.8 


0.0 


7.3 


6.0 


-1.4 


Offers Based on Referrals 


(0.8) 


(0.8) 


(1.1) 


(1.0) 


(0.9) 


(1.4) 


Sample Size 


419 


425 




640 


703 





Source: JSA demonstration follow-up survey. 

NOTE: The impact estimates are based on differences in outcomes between the control group and the treatment 

groups. 

♦Statistically significant at the 90 percent level of confidence in a one-tailed test. 

♦♦♦Statistically significant at the 99 percent level of confidence in a one-tailed test. 
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Furthermore, the particularly high rate in Florida is also not surprising given that the JS and UI 
offices in that state have been merged into the new Jobs and Benefits offices. 

In addition to increased contact with the JS, the SJSA treatment also increased the rate at which 
claimants received job referrals from the JS. In D.C., 24.2 percent of the SJSA treatment group 
received a job referral from the JS. This represents an increase of 8.7 percentage points when 
compared with the rate for the control group (15.5 percent), and this increase is statistically 
significant at the 99 percent confidence level. The SJSA group also had a higher rate of job referrals 
than the control group in Florida— an increase of 3.4 percentage points, which represents more than 
10 percent of the control group mean. 

Although SJSA increased contact with the JS and job referrals from the JS, the treatment had 
no impact on the percentage of claimants who received job offers based on the JS referrals. On the 
contrary, the two estimated impacts of SJSA on the probability that a JS referral led to a job offer 
were zero or negative. This finding suggests that increased contact with the JS did not bring about 
greater re-employment that could be attributed directly to JS referrals. However, the increased 
contact with JS may still have been indirectly responsible for greater re-employment. Claimants 
could have used their contacts with JS to build job search networks that offered job opportunities 
outside JS referrals. Earlier chapters provide some evidence that the SJSA treatment had a positive 
impact on re-employment, but it is hard to know what role specific JS services or referrals played 
in that impact 

2. Individualized Job Search Assistance 

The USA and IJSA+ treatments had impacts on JS contacts that were similar to the impacts of 
SJSA. Estimates presented in Table EX.4 show that USA increased the reported rate of contact with 
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ESTIMATED IMPACTS OF INDIVIDUALIZED JOB SEARCH ASSISTANCE 
ON USE OF STATE JOB SERVICE 
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the JS in both states, while USA+ increased the rate of contact in D.C. The estimated effect of 
USA+ in Florida was also positive but not statistically significant. 

The individualized treatments also tended to increase job referrals from JS. In both states, the 
IJSA+ treatment generated large and statistically significant increases in the job referrals from the 
JS. In D.C., USA+ increased the probability of receiving a job referral by 8.7 percentage points, to 
24.2 percent for the USA+ group compared with 15.5 percent for the control group. The impact of 
USA+ was similarly large in Florida, where the increase was equal to 10.3 percentage points, from 
28.9 percent for the control group to 39.3 percent for the IJSA+ group. 

The estimated impacts of IJSA on referrals were also positive but not statistically significant in 
either state. In D.C., the rate of job referrals from the JS was 18.4 percent for the USA group 
compared with 15.5 percent for the control group, a statistically insignificant difference of only 2.9 
percentage points. Similarly in Florida, the referral rate from the JS was 32.7 percent for the USA 
group compared with 28.9 percent for the control group, a statistically insignificant difference of 
3.8 percentage points. 

Despite the increased referrals generated by the USA and IJSA+ treatments, the treatments did 
not increase the job offers based on these referrals. The estimated impact of USA on job offers was 
negative and statistically insignificant in both states. The estimate for USA+ was negative and 
statistically insignificant in D.C. and positive but still statistically insignificant in Florida. 

These findings suggest that the individualized JSA treatments achieved the objective of 
increasing access to the JS and the use of JS referrals. However, achieving these objectives did not 
also generate increased job offers. Estimates based on the combined individualized JSA treatments, 
which are shown in the final column of Table IX.4, reinforce this conclusion. The combined 
individualized JSA treatments increased contact with JS and job referrals from JS in both states. But 
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the combined treatments did not increase job offers based on JS referrals— the estimates in both states 
are negative and statistically insignificant. 
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X. THE COST-EFFECTIVENESS OF THE JSA TREATMENTS 



In this chapter, we combine estimates of the impacts of the JSA demonstration with cost 
estimates to assess whether the benefits of each of the three JSA treatments exceeded its costs. We 
assess the benefits and costs of each treatment from several different perspectives. For each 
treatment, we compute the net benefits per claimant (benefits minus costs divided by the number of 
claimants) and the rate of return per dollar spent (benefits minus costs divided by costs) to claimants, 
DOL, the entire government, and society as a whole. The net benefit and rate of return estimates in 
this chapter measure the cost-effectiveness of different JSA treatments, and can help policymakers 
determine the desirability of the treatments. 

The cost-effectiveness analysis in this chapter uses estimates from earlier chapters on the costs 
of operating the demonstration and providing JSA (Chapter IV), the impacts of each treatment on 
UI receipt (Chapter V), and the impacts on employment outcomes (Chapter VI). For each treatment, 
Chapter IV provides estimates of the costs of the demonstration and JSA services; Chapter V 
provides estimates of the reduction in UI payments that benefits DOL; and Chapter VI provides 
estimates of the earnings increases that benefit claimants. In this chapter, we weigh the benefits 
against the costs to determine the extent to which the three JSA treatments were cost-effective in 
D.C. and Florida. 

Based on the results presented in this chapter, we come to the following conclusions: 

D From the perspective of DOL-sponsored programs, none of the treatments were 
cost-effective in either D.C. or Florida. The reductions in UI payments never 
outweighed the costs of the demonstration. Furthermore, most of the additional tax 
revenue went to other government agencies or departments. 
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G For government as a whole and society (i.e. government and UI claimants 
together), the treatments were cost-effective in D.C. but not in Florida. These 
results are largely driven by positive earnings impacts in D.C. and zero or negative 
earnings impacts in Florida. The earnings gains by D.C. claimants provide benefits to 
the claimants themselves and to government through an increase in the tax base. 

□ The results cannot conclusively identify which treatment is most cost-effective. 

A. IDENTIFYING THE BENEFITS AND COSTS 

Our analysis of benefits and costs focuses on three parties who were affected by the three JSA 

treatments: 



□ UI claimants 

□ DOL (SESA, which encompasses the UI system and the JS) 
D Government as a whole, including DOL 



After analyzing the benefits to each of the three parties separately, we consider all three parties 
together to evaluate the cost-effectiveness of each JSA treatment from the perspective of society. 



1. UI Claimants 

The JSA treatments were designed to hasten the reemployment of UI claimants. The negative 
impacts on the duration and amount of UI receipt (Chapter V) and positive impacts on employment 
and earnings (Chapter VII) suggest that the treatments were modestly successful in moving UI 
claimants from UI to employment. The impacts on UI receipt and earnings had opposite effects on 
the incomes of UI claimants: 

□ Larger earnings raised claimants’ incomes 

D Smaller UI payments lowered claimants’ incomes 
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Whether claimants’ incomes rose or fell depended on the relative sizes of these two effects, which 
we estimate in section B. 

The increase in pre-tax earnings induced by the JSA treatments does not fully account for the 
benefits of increased employment to UI claimants. Government taxes capture part of each additional 
dollar of earnings. We impute the additional income taxes (state and federal), FICA taxes and UI 
taxes paid by UI claimants for each additional dollar of earnings. Adding together the federal 
income tax rate for the lowest income bracket (15 percent), the state income tax rates (6 percent in 
D.C. and 0 percent in Florida), the FICA tax rate (approximately 8 percent), and the UI tax rate 
(approximately 1 percent in each state) leads to tax rates of 30 percent in D.C. and 24 percent in 
Florida. 1 Therefore, we assume that for each additional dollar of earnings, UI claimants pay 
additional taxes of $.30 in D.C. and $.24 in Florida. 2 

Furthermore, employed people often receive fringe benefits, and the generosity of these benefits 
is typically higher in highly paid jobs. Therefore, we impute the value of the fringe benefits likely 
to accompany .each additional dollar of earnings from aggregate figures provided by the Bureau of 
Labor Statistics (BLS). 3 According to the BLS, the average cost of employing a civilian was $20.29 

'The UI tax rates faced by employers are fairly complicated. Therefore, we use the average UI tax rate (total tax 
payments divided by total wages in covered employment) in each state as an estimate of the UI tax rate. The average 
tax rate is slightly less than 1 percent in Florida and slightly more than 1 percent in D.C. For simplicity, we assume the 
UI tax rate in both states is 1 percent. 

2 Other reasonable ways exist to impute the additional taxrevenue generated by the JSA treatments. However, as 
shown in this chapter, the conclusions resulting from our net benefit analysis are not influenced by seemingly large 
changes in the tax rates faced by claimants. 

3 In Chapter VIII, we show that the treatments in D.C. had a positive impact on the proportion of employed 

exclaimants with jobs that provide health insurance and pension benefits. However, we lack measures of the 

(continued...) 
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